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? r >y rasa*, ?7^ryifAMx Sr-/r- lx xts^ 

[11*113] ifia^&3&\ wj&igii. ¥#si§tSL y 
t-£>4 £ t *wafc-r*m«^ i tiaiwa^ea-^ 

soit5£<5o#Bg^^-tf £ k Sr^mt-rs 11*^3 kibko 
[|f*ia5] vtwis^\ isiisti./t^ 

*£W3M-4SHWS3i*SWR 0 . futBfefll* Mia? 5 

4fcft<0ilHS-**tr £ k ^ftmt-rsif *IR 1 KJBttO 
a y tr j. - fWffl. K> ^mmik . 

[ If 3<if 7 ] fftav- 7 7 >y TWfg^' . madMPtfiR 

its tvttr-* fc # w # UEh.* ± a 

[ liSJii 9 ] frta-7- ? 7 <y twis^' , ttr ia-r ^ x t- 
-x^m % m^mm. & »*ro * fc *> <z>#** * tr £ 

tiffin. 

[ft*ii i o ] Miev-? 7 7 -rmmtiK twa-r^x 

hJ^iimfr*fclb£ffiSttiWmtt\fi> £ k £ 

# it k ^-^>f*3f<ia 9 tiattco 3 v t a - o »imm 
*. 

— ? 7 >y 7WM^~ i>'£SftK4 Xf -y Tk . 

xt -v 7 k SrAtfXf- -/ 7 &9tffi-& £ k lz 1 -?t 
* #tf £ k Sr #ffii k -t 4 3 y t * - 9 ISJR 9 nTffi 



»9WS £ t ^«Mk t-§lf*il 1 1 fc3B«W3 >tr A 

[ff*iI13] HirlEftf^*^ 
£Hri?£ t SWati- 1 2 CiE««n y t A - 

?7^rylf>^xt (iM^r^ * 7 17-7 £«7 H 
tz^y^ryhfy u x^z xh -f - y < —>v y 

WtSv-777 TWm^- V * SOTX 6 X 7 -y 7 
k. 

If *»^s> A73 SrSttK^ Xf -y 7. J: t>'l5iaA73 iSrt 
■r— ^tJ iWB»ffl<?33tS£«)ftSt, luia^ 7^ 7> b 
-r ^ ^Xt {±M5: h*-yY r 7—7±.<r>T} i l'X\ l zfo%>U 
BWw ■< fc iS&T 4 x^>yTk^-irtf£k^#11k^-4 

tif*ii i 5 ] iwfEx-?£Si<rf stofc, friaATj 

Stth-T tiiiaf 1 '-^ IMMWi^fy 7§r $ 6 t*tri 
k fflt ^411*51 1 4 fciB«<0^. 

imem i 6 ] ms^«^t^G»r£8Kns%» 
fitti, MiafeasrMia^^-f r> h-r^'-f x±ts> 

h 7 A -IV H tBffitftJti Xf'y7^§6 t=*tf 
£ k &#Uk t"4ff3f<ll4 1 fcaatt^^rife. 
[f**3Sl7] ^7^7yb/^-^S>X7A4i^7 
5>fryif^ X-tHtr-T-g. fz#>cr>~?- 7T-y 7fi 

- 7 7 -y 7Si*t, B»ffl(^M£ * fcMdSTffifc^rr 4 * 
7y'x^h ; &T^^-r^^tj , £kSr#mk-r4 3 

[f»*H 1 8 ] IfflERSfc&r, fF^B», 1211, 

1 mi&btoWrhhZ. k Sr#Hk-r4H*H 1 7 l,z 

mm 1 9 ] ffismmK mmmco^mcom^^ 

JROTTflBK*. 

[if*il2 o ] BHawt, fgfi&k'co-fdf x h3t* 
?|J & W tJR o T 4£kt;raiS-r4£kSr^k-r 

[ff*ii2 i ] mria^^x bjt#9d*fl!i^)aMEffife&» 

^ St m 4 H 2 7' l J x 7 b -r';P * S 6 tzittt £ k 
Sr#mk-T 4ft^l 2 0 tcia»^ a >- b° a-^KR 0 «T 

yATyy^r) U XT^ff -t 4 Tt^cov- 777 711 
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[ft^2 3] mZ-?-7T-yywmt>K HTML. 
XHTML, c HTML. XML. ti X tFWM LCD o *> 

i o**tr£ t 2 tiaRfw ye 

n y e * - ^jr 0 nrtg^ft . 

[8W&H2 5 ] fulfil-? T vT3Bj6*, ^jKSfUt 
[000 1 ] 

WebTttfflWS^8»=HH-i. 
t 0 0 0 2 ] 

-v (P IM) ii:"«/Mnyta- r-f y^-f^-f x. 

i-4fc-ah.T, £ ft <a<a^A>f comfit, ft±U 
tei-oTJiflteSftS. fet ji{fSfcWe^<o^<{±, -f 
yf-^?l T7 J tX33i&?'y*7 t J> fifflmv 

t> *) , s^ctitx, misses: ir^fflAW tan 

[0003] zixhcoay^jL— 7-4 y?"7'M xtfi. 

-f y^-*-y hC07"5^^>^"tffiffl?fL. istdfl&w 

H^_y\'/^7-f r y hr-*'r7-?-vTmmzfi& z t 

izmz-T. 'IMSSrrjytji.— -f-r yp^f^ xtzAJl-? 

»4?S:#6, ift^xV^fxfi;. ft*> 
3KWl^ J 3 (CTS 4 RR 0 ^ffiTifc 4 £ b tmt L 
< . 3 y b° a— f - < yy^fj^ x<r>j\y : Jy7±.xiiM 

K.-yhJc¥*m±Lt:#?yt~t& OsteO^f— F 

[0 0 04] ja^fii. Voice XML (voice 
extensible markup 1 a n g u a 

s e ) zmm-tz. xo^pt'-9)vt>K mmco^im 

m LX >f y * yl3 y^ y 'y tCT^ -trXT'^ 4 £ -5 

tetiiSftT^Jt. ^r-^T-^-f-vm. 

yht-A* (it We W-A) (±. Voice 

X M L A y 9 — T V 9 £ it L T ? 5 -f T y V h eoWR 

&Nmr&. we b-^-jmzmzm^Lxv o i c e 



XMLF*r J L*yh%fEm-&Zbti**imX'$> , 0. Zfi 

t^Ra^yMi voice xML-f y^-ru ^ 

CcJ; o T-x- #'1 & WiS J; 3 fc*5a S ft 5 . «f 
«. W^ISH^^-L^W^^vy KSrffifflLT. Web 
Sr^-t'y- b~tZ> Z b h . 
[0005] 

[5&BJ!* s l?^tj;5i:-Tl»iili] Vo i ceXMLIi, 

oHB Vmt . ^yiX^ijrifcJ: TOfX ?'JT1 
^&HTML (Hyper Text Markup 
Language) 7 n-fitf/l'fci 3 ^> tfOTii: 

=5rv\ tfU*. Vo i ceXMLIi-it, mzWSS^ 

~mz3--*ffrio$mztifzmMt,±. yx^A&tz 
uttv $-—y 3 y<DmMTtzw.frti& » f?? 4 ij>v 
jl—w yf-yx-xitm&ztLT^hry-iryh 

t-^MItffifflt^ 1. 77'J y^-y a >C , Vo i c 

e xML*aaaa»atrta, v o i c e x 

Mlffl«ti7)tHTML (4fcJ4raab5fc«0) Sr^ffl-T 

4 h cox-fa *) . -eft-eft^'M^rs y v-mw^Mzm. 

[00 06 ] LJt^'->T. -fyj'-^yhiifW-A' 
/lyATyYT-^l^^X-^mW&W&thvy 

tzmm ?iti,7^f^ftfej: W6ra*sa#-r a i t 
v— Mi. p 1 M^mm^b'co^Mco^y^A— 7-4 > 

[0007] 

[0008] i<7)J|i;5Sgp (extension) . fed: 
U f *l6BH«?5fl&c0SMSii& ffiffl-C'S SitT". ?5^ryi 
f'AMXli. A*T— /H«S:^rtSWe b 
if — ) \frt>-?~7T -y TWm^.— V £S Vm h tzibCD-fa 
^mft-tz Z b h . vf? 7^7yl r'/SM 
X{4. fW7 4 — /H ; ic|SBjS"^"4-3- — tffrhJ^fl^ — ^ 

& . mm iz&mm c u n es^BBH § ti^ists 

[00 09 ] Igft^-^'ti, B»*Hff^6fcftfc:» A 
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[00 10] 

4 !T£ » * <0T-*-r ? -CUKT § ft — *KW 
^rayti-f -r vy*riM TUcov^TiM^ftTf A** 
WtXfoh o . m 1 t , x-? WfVs^ x 

(PIM, P D A& b" ) <7)?^®W 3 0 IZTjk^tlX V * 

ft. fc*£U ^WMli. filTCfttittwa^ea-f 
4 y y* T> ftffcWtw x. Si' . A7J # ^ yffl^r 

ftl>cOis&^ L^Vs^x^ttfg&it^yy^ 

T-3^y* if-vOSEHti, *sj*0«^ Lfc»SW*-r 
— *05SK?fc i -?TPfijE§#xft £ k fcSEfr* fcOTIi 

[ooii] c f—?ism&m j fj*<i x3o owswjbib 

y*3 2 3 fc^tfltfcfl*Sfj|BS 

iEttW-r -f X T V A MM Sr ffiffl-f ft r > X T V 4 3 4 

(i . 7 -i ~)V F * 3KK"f - ft ft tcfSS § fut ffiSfii 

T , T'W X30 ±t^b*y-^ g y-ffltfO 1 fc{±« 

>f-/K n— 5^ri;<50flfitf0A^J^^-XASr^t-rft^ 

v*^4££fi0«J(S«F5rA***tt£ t ft o 
[0012] i^tcE!2 ft 1 . ftmf'U X 3 0 

*^«iHtl«H*W«iacHj6*^*iiT t^ft . **2&3 
i-.yh (CPU) 5 0(4. V7h^iTW«iK*ll 
Mrt-4. CPU 5 0^f^71/>f3 4(:fe^iltV^ 

tiXU^'yy-i 7?7^3y#T'a7M34(:^ 

sitft. xe-*4 3ii ifnBffi^j&swwifcftfc:, 
u 5 o t^-rft i t ft . j.— fti r>xy*7v 



c p u 5 o t mirmzm&znttwmmm. >o /» 
^7y/A7?wt'j lEfi^a s 4 t flgtft § ti ft . 

y>y^T7*zXs<^0 (RAM) 5 4(3, CPU 5 0 

t £ otiff §fLft^fflt7)M^ffiie'ii«i. *j iz/u^ 

x y-ffi^r if O-H^r- ^M«E««*!iflW- ft . «)iS£^ 
ffl^'J (ROM) 5 8fcfSS^tLft„ ROM 5 8 

gi«RAM5 4^0n-f>y/) STffflffl-f-ft, -rVs; 
•f7l«^l/-f ^ y?y7fAy7l7i7J|gfft 

-rftvtfttffiffl-rft i fc fc-c* ft . 

[00 13] RAM 5 4(4, 77^-^3 770^7 
A £fg#rT ft coWSffl $ixft p c h h' =?A 7'co 

mmizmuLtzjfmx\ ^-bmcDummt lxv&wi 

ti%^»ft&X*V rt^S^ft d t fc*CS» ft <T i: t« 

[0014] mmm^-ii, c p u 5 0 fc^^^^s 

h 5 y is-/* 5 2 Sr^- L/T#»7*' ^7ti otSffi 

x-^^fiS^yn-h'-TS^ftC t7y 
a ycornm^ >?~y 6 0 Srffffi^-ft zti>T% 
ft. U^t^y^-7i-7 6 0(i, fcfciif, * 

^HiOaft^VsM X£ *tf i i; ^T'* ft . 
[00 15] §1^^X3 01^ 7-f?n7t729 
fciUfTtn^fy^^ (A/D ) »3 7, *<5>V 
C:IB'lfSS 5 4 ttSttS#iJt;J-7° y- 3 v<7)Sl7n/5 
A DTMF, 4^*^, y'xXfr, t3t(irJ>' 

*JA/DSK»»3 7tcJ:'5-CT : 'y ; ^,'Wt$ti.ft. 

it7°n^5 A(i. ^t^^r^**JK#-rft fcft iiz , 
fci*ttw*aaiiB*iiff^* i t ft . w?»-f-^ 

6 0^ffifflLT. lilT*C#^tia50r— dff^ftt 
tifv! . 'J b tSlt-F-; 2 0 4 (CfeiM SfLft „ 

{iB3itW) . ^feii:UWebt-A'2 0 2 (H 
5) Ce3S^-&fc«>fc:8P«|f f A>fX3 0(CMS*t, We 
bHf-vs'2 0 2fcj:WifAMX3 0te?yj T>b 
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M£H£t^X3 Ofcitf/ifcliWe b+r-A2 
0 2 CO o <thl HzWtZ 1 £ . fflflt 

DTMFf - ^, yxXftf-^, fciOTIf 
Sr m t 4 3 fc*5H"$-«. i t 5 . A7J®«tiE 

[ooi6]03ii *^«IS8 ooMiWiftiU 

ij^PHHT* €IS 8 0 C (4tm x 71/ >f 8 2 is J: 
yr^-ys; y y 8 4 3&%4*t4 . —©tell 2 coif ftHttBI 

3tfMH<90R£K*^*£fcj&*T&6. *rh;t(f. 02 

[0017] l?M(?«»»4fcl*Bf«3^ea-'r-f 

r^-f xtJni.T . ^^MB* 1 , tfflfx^ h 773 yt° 

Sffl "TigTA 4 £ fc i> JfflWS V> . i: X (3"*?feBJi 
(4. #fWt«B*«l|!llRS*utJL-ifAJ, 7; p^r^^_ 

yea-f -f V? 3 ?** X{ZKhX% s 4 -5 \z~th i>co 

[0018] *3fc(W«4. fl&co^R<oifWB4fc{*f*Sffl» 

<>IMPWr*4. *HW^ffiffl"*-4«fcifSMr. 4< 
^^ayt^-f^ y/yxfi, fc4t>V 

S ?t ti 9 >y 7° h v ~Tf) >M X . -TVl-f-Tn-fe 'yty^f 
A. V-f ^aTO-fe'yt^-XyXf A, -fe y b b -y 7 

#.y?x. ro^7A9rtg^«iWB«^«B Q B , *>y b? 

— 7PC. 5— nyfa- y<--f y7l/-A3yh°a 
[0 0 19] iJcfc, H4fc3^tfcilWB3yejL— ^ 1 2 

2 0 ijffJS^ra^tf a— - f * >-2?Ti^O— flajKaft^'f . 
S-*i®-rSfc<?DT{i^rV\ S ^tcaytj.-^ l 2 0 

S**^**^ 4 ^coh LTW^Sni. ten. fcfcv*. 

[0 0 2 0] 2f£^BJ(i, 3>-f A-^ioTHfTSfl 

§ 7n ^ 5 A ^' jl — }V% t'<7) . a y b° j. - ^ Hff «Ti£ 

^^-mm^mx-mm-th z t & . 



X'fo h . iMJcn y b° * ~r ^ > ^"ii«T"{S , 
^^'jL-;l-5r, ^^UIBItf-V^X^tf, o— ^;1-B 
J: X/ U * - h cojtfjyj CO ^yta-^ iHUEtt 1*1 tE»f 
&Zb b^mX'fo h« 7P/5AfcJ; =J *. ~MZ «fc 

oTSifls *is ^x ^ tov , tUTxm^m^xm 

[00 2 1 ] HI4 5-#HS-f-§ i: . nyta-^ 1 2 0C0 
«J«S*t(±. J8SI3-- M40, y7WtiJ 1 
5 0. fciV'yXrA^t U**tf«^=3r^'rA*ffK 
gS5»la- 7 b l 4 0tg^tl.y7rAAXl4 

Xr-Ays'x 14 1(4. ^t'JA'xtti^t 'JSM^ 

coy UX h 5 y 7 -f- *rCD o fbCD^tlfrX'fo otiV^ <yT| 
SrWSt. id t7^T-dr-ry'^ J r«S. Indus 
try Standard Architecture 
(ISA) A'X, Universal Serial 
Bu s <U SB ) „ M i c r o Channel Ar 
chitecture (MCA) ySX. MM I SA (E 
ISA) Kx, Video Electronics 
Standards Association (VES 

A) n-*)WU, fci^-f— y^xthmiti&P 

eripheral Component Inter 
connect (PCI) ^Xti^itL&tf ^ Zti^l/Z 

m%.-£fL&i><DXliZc^\ ay^x.-? l 2 0(4. ftM 

f A-^RKO^TBBKfMi, 3>-b° A -^l 2 0m 
•fex-T S ; t cot-# 4 ffi&cofiM "Tfig^ft-t' ^ oTi 

t^ig«*coi^7T^^o 0 M^^ifst. 
\wmxnmzixh. mm&xx&Fmm. w\-m 

— ^iaii^E*(-(4. RAM. ROM. EE PROM. 7 
5 7 y t 'J i tliffi^^t 'J Wis CD-ROM. 
fy-?)VW>m'T4X7 (DVD) 47ci4fl&C07txVXy' 
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ffimzmm-&cotz&m-*imx'fo ^yhw 1 2 0 
9 BSMttiiM&immm^t tit t>K ztih 

[0022] mmmziz^ mmmtwrn 

cmm^^-x^K^mm^-^m ^fpx\ tj y t° 

^tAc^fe. FR, *7hSL iji^ffioiElS^^fo 

[0 0 2 3] ixXr-^^^EU 1 5 0CI±. MffX^JSW 
^'J (ROM) 1 5 lfcit^^^^T^-bX^^eU 
( RAM ) l5 2&fcV). W¥m&£V/ttdZ^W% 

iz&iL^m^—i-y^ts. m^Atii^J^Xr-M, 1 5 
3 (BIOS)(l ISWCiiROM 1 5 1 tfgft*l£ 
til>„ RAM 1 5 2(iAMWWi. flPB^T^-fe^nrtg 

x-fo&fr, hhvm/^^if^zMmm^-y hi 4 0 

£ J: TiWt'&S . r — ^fc itf/S tli7o/5 
5 5 . \WT7°^9'y^'J^~)Vl 5 6 . HXVy°X39' 
[0 0 24] 3yt"i-?l 2 011 ffiCDj^LnTSa/ 

**tri Htl^. @4 Ttl JKfl- L^t£<7^fffltt 

?i72t <7)mxtm *ottz t,zw&& zno atstr < x 

9 F7^/l 7 1 . i^fcCD ROM^d±fl!l<7)3fc 

^^flc^^amL^r^^wsMtMef'-f 1 7 6 

t 0 * JtJi*S* * fi 1 5 jlfrr -f X 9 Y 5 -f 7* 

iz-tw^\ M^w%mimntx'mm-t& z t co-ct s 

a— ^IB1i®#:t<i, l^f-7*-fe7f < yy-vi/x. 
T— 7°. y'J'/Wf-hRAM, yj'yWf-bR 

1 y^-7x-7 1 6 O&fCOK^L^IgTi^lM 



-y x.-X^i\-LX y7fAA7 1 4 1 tlgffiSfu % 
Mfa^ Vyjyi 7 lUXVytr 4 X9 F^/l 
7 5(1 ASS^tl ^y^-7x-X17 0^t^l 
"Tt^^U -f y?-7x-7tJ;-2Ty7TAyU 
1 4 ltffiKSfLi. 

[00 2 5] JJE"Cst*LH4fcij*Lfc K5>f rtJ XV 

% emstf ft fefifc ^ y tr A - * iBttttftii , =j v t° * 

Jg. 7n/5Aty'a-/K is 1 OTffi^x-^^fa'lg^ 
tSflWS. tztmmAXlt, ^-¥^4 X9¥yA 
7*161 jE^^I^— r -f y/y^fA 16 4. TT'J ^ 
-•/gyTo/^Ai 6 5. flfjWT'u^Atjyj.— ;P 
16 6. t$ XVy°n9yJ±f—9 1 6 7£*§ttLT^£ 

-r -f y/yXfA 15 4. 77'J ^— 3 >-7°cry*"7A 
15 5. ftfeCOTn^'^A^Ev'j.— ;H 5 6 . jolV'7°0 

(5. ^-7^//^fAl 64. 7*7° 'J 3 
yrn^Al 6 5. ii?)707 7Atyi-;H 6 
6. fcitfyn;'5Af-^ 1 6 7tc{±. fc i>M 

[00 26] *m. ^f-t;-F182, 7^D7 
tyi83, fcitX777. h7 7^*'-;l/, 4^{4^ 
7f^7 yf-f ^f^7 1 8 1 ^rfcO 

ATlxV \M X Sr^- LT . 3 7>- H X VffiM & 3 y b° A 

-9120 \zxin-fh z. t **trs 4 . fanx-fiTrW a 

^tilMifi^ATJr^Wxii. y7fA^'7(;^§fi 
^a-f A^j >f y^-7i-x l 8 0 ^^LtJIla- 
■y h 1 4 0 tffijiSftS^^l^l b;^- 
h . y— A^— h . i«iU niversal Ser 
ial Bu s (USB) %b'CDfficr>-4 y?— 7i- X 

1 84^J^(if(!l<7)^^7°OT : Vxri^-fT ; 7^-fXt. h* 

y-9—y 3,—x i 8 53r£'<7M y?-7i-7^ 

^•UyXfAA'Xl 4 lfcS^^^xS. =?yb°jL-^ 

{i. ^-^tcjax.t. aJTjJilia^ y?-7x-xi 8 8 

^irLTSM-rs^^w^sxt-^ 1 8 7 & it/7° 

U 1 8 6 ^rii'cOffi^JliafliTJ^S'l xi^: t 

[00 27 ] 37b^-^ 12011 'Jt-b77ba 
1 94%:bCDl~o£.fzl3.mkC0V : £-bayh°~L- 

9^wmm*®m^-& * -y h ^-^awtwf^riB 

TftS. ijt-h3yh"j--^l 94J4. yN°— V-i-iVn 
yh'x.— 9, ^^>''K^;^'H'•f7 > ^lX. Hf— y~\. ^. 

17-7 7-raoTK. AM6^j(c{i 3 y t a-^ 
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1 2 0 fcB»UTJJETj*^fcWR^)iK *fcI«-^T 
*#tf. H4^S*ifci*saaKfc:li, n^/lx'jr 

^•y!7-^ (LAN) 1 9 liSi^'f b'I!i7*7 
b"7— ? (WAN) 1 9 3^ttil^K McO^-y h7 

[0 0 28] LAN^v ^y^WWCffifflSfl* 

nyt a-^ l 2 oi±*7 iv-^^fy^-^x 

— X$.1Z\ZT¥y? 1 9 OSr^-tTLAN 1 9 
M£;fX-i>o WAN*7 !- V - >• y^tftrffiffl $ ft £ if 

nytW 12 0it AIB^IitfAl 9 2, 
ifcfi-'fy:?— *>y h&^CDWAN 19 3^^-LTffi 
fi^5l^t"'S.Jtft^ff!l<73^S?r-^tf 5 t-r'A 1 9 2J4F*1 
^I4fe(4iWJt3& f WC& 0 , jl— if A7J-f v^-7 

x-x i &o&mim<nmw£mm*-ftLTi/x'rJ>.j^ 

XI 4 1 {zmWtth Z t tfX'% h . 7 

t4, rJVb° A -^ 1 2 OtBljSLT^Sft^To^A 

Tlill^r— h77'Jt-^3 y7n/7Al 9 

-hnytji-^ 1 9 4±fcflBftL,T:^*J:5t*S*i. 

[0029]i5tl *fSWCHK"rifi=ar, We b<- 
XfgftfflCDT-^ x^^-v 2 0 0 fcij^Brfc* . HK 
fc. We b-tf-^'2 0 2tf§S*|$fL7tWfga±. Wbr? 

7-j>?T : ^x<DmcDBmi>mLx^&) , a^ia* 

fSSO^LTTy^XT^^fc^trCTfeD, ffffi 

*i4^fcJ£3Srlyt*IS8 0j&*£j£1-S h-y(;i4->Tg 
We bt-A'2 0 2^^«ffifg{4. *fi-f 

[0 0 30] §^ClS^tt, T-^Xy*^-v2 0 
0 (4, W »**r A>f 730t «^Bii&fl6ffl^-*«g8 
00^^£^LTf#£ft&4ofc^£;ftX^& 
t>K #— 0>BMWw* 2 0 4 (4 ff *> £><JD*- Feo»ft^ 

0 0J44<ft]4ft^V-?T>y7 o Wfg (tttffHTM 
L , XHTML, cHTML XML. WML^r^)t0 
BSRasfcttffll/ClWWS. Lfc^ot, Webf-A 

2 0 2lCfSW$il3tfflfffilw(i. ;ft^<50^-^r-y7°g 
ffifcA6;ft.4±<jai6;ftfcGU l7fi^*ffifflLTT7-fe 



JMiSB^ffiffl-rS^-C, We b-f-A2 0 2W- 

[00 3 1 ] — j&fc, f^4X3 0(4, We bt-A'2 
0 2 HTML^-y, X 9 V 7 h & fc* * 3Sfr 

iftf V*/WtT4 - a jfi^S Sffl*r- * , £ ii.it 
ffii&coZ. 3 IzfU x 3 o tc4 -3 T^MiWOTWl 
§ftT v ^ #^»waw* , OTgampKffiffl-f e 

^^Jti^ali^x/^f^fcftfc, BMW--;* 2 04(; 

. fg^-A-2 o 4comma^< vamevi 

fS^toCfAM X3 0t~M£ft&„ fg^iV'ffifflS 

fLX\ >&H}^Ziinc0777 4fl)V3-— J ¥A >^-7x 
(4, *W«*£Webirw'C2 0 2fc38©LTS6fcM 

[00 3 2 ] H5^§ftSJ;atc:, f^x3 0> w 

e b-tf-^-2 o 2 . o 4 {4, — mz 

^7l7-?2 0 5 t^LTSSttStlft* 1 . Waft 
(4-f y^-^-y Yt£K<nv4 F^OT^v hV—fi-ft 

Lxmmt?tix& k> , t=r f uxj&mmx$>z> . 
fcBHes***ea»4*v^ **w^m^«4. webt- 

A'2 0 2 ^mti-tf-^' 2 0 4 Sr^-tj'^(4 : &V ^ CI c?)^ 
^T"«4 .Web ->f — y S' 2 0 2 W^-f L> y ^14 . 
* I BMW^^'«2 0 4<?DaBKrp^S*jai*4esSSr< , KH 
^ixTUS 77U ^-y a y CM^ *t I. t &*m 
X'h h . tfL-5fg^-^'2 0 4{4, *3*TlSftLT^ 

7i7-?205 i t immx-fo o , ^ti^ 

4^T, Webl^— A2 0 2TSR§*t43SMrLfc:, 
5eW4i 4 VStft-r-l. <! 1 4 . HTtt t & 4 

dt, webt-A-2 0 214 ttWfc^^-f ryFWo 

~?—9T'V 7HXV ( X 7V7h i 1 36*77S s 

t-t'Jy **tr i t fc -C § 4 . fl&^IKBBWT 

{4, Webf-A'2 0 2. IIii-f-^'2 04 , feit/y' 

^r. y*54ryF{4lg^--VN'2 0 4^tfifc^'-C# 
5. H«C FJfat'fetKf, We btf-vs'2 0 2fc4V 
IS^-^'2 0 4 Sr#— «7yy fc«*att £ t 

•So 

[0033] *%Hgo— ®«(4, /7-iryifAM x 
"f^^ATJx-y'SrJRff-r^ 4 3 IzMf&Ztitzm 
m3Ztt&v-?T>y7Wm^-i?Zy--^frt>%:tt 
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TMF, yxXft, ^fcjjrr) 

*j i tx isaa-ft tt 6 sas £ ? 5 >f r y y tpkwmiz h 
-f r>- i/t-^yxfATAAf-^ &imt&iz#> 

fWSrffiT* h . ftDJIMKM: . ? 7 4* T > F /-f — ' n* isX 
r- ASO ? 7 A T > Y 7-V U x XWn~t S tdsb<W— 9 T 

v :mf§£^rf & a ^ t a - ^isr o fwtib -5 

[0 0 34] OSr^-L^We bt-A°2 0 2^.0 

7?-feX(;(i. miS8 0^^^^^{iMllffilg^'/ F 
T7— ^2 08^)SSlb& t *4*t» £*U6**R8 0£IIS= 
#^-h^x^ 2 1 OtSSBM-S. y-h^x>f 2 1 0 

fi. «1S8 0£«S^R»7'5'^2 1 2 ttsasttft. « 

fg^y^-f 2 1 2CI1 «fg^>-^-7i-Xi3j: 
t>'W^7'7^'2 1 6 .X-r 4 7*^-^2 1 4 

2 1 2ttHTML^-y/X?'J71^t"&Webt- 
A'2 0 2i^fltlS„ SfefCfi^Sr,! 2:^. HTML 
^-y/X?'JTMl fA"^X3 0£*gffi§*t&HT 
M L^—is/X? VThb |B|«c?)JFMT"$>l> „ - tOTfsS 
T(±, We b-f— y\*2 0 2^x^4X3 0 J;lXfl;f£8 

o tai^fclhif- Ft-^g^&v^. £st«iffi2P?)G 
u i ^7^ryh^su«t^-h-t«.«est^rv^ tf 

h„ 7*^^3 0 tPHKfc. t3S8 0tJ:-9T 

m^tih^m%^frh<n^m%mt>K Yv-y 2 

0 5 4fcIi*fflHJ«2 0 7<0^'f*iA»&^L , C, tzbx. 
KTCP/I PSrffifflLT. ffi^T^I^ 1 6*^12 
lt+J--A2 0 4tjggt§;fl&. Webt-A'202, IS 
»tfw«2 04, iSi.Xfm&fpr7 , 5*7 J ?2 1 2t±. H 
4 § sftjt ifJlfflx'X ^ly73yt"a-?i JfOffiR 

So 

[ 0 0 3 5 ] fcrtfU DTMFB»* t ffifflS*U*a\ 

f-fTt-A'2 1 AX'mf^flhc\ fcfcffiKSfufcVv, 
B^&i-ft.f£. D TM FiSH^f -f 7t-A't i 

[0036] M^=t 3 fc. *lffiB^>-JI«N±, ? 54 

s'r-^ f ? f - * iztmz mm- h mms 

i.-tf/$.td<tt7 i J^?Y*^fr>%>tztbcr>, HTML. 
XHTML, c HTML, XML. WM L&iT «7- ? 
T -y 7°#|gO#3Sgfl£^-tf#\ £ Jt(iffi&<9ffe<9 S GM 
L^ffiM-7-^T-yT^fiiTV^o £0>3n*TW:. # 
Lit7-^f^f^T'ffiffl§fl§fA^We 



[00 37] HRfc, Mff& itf/lfci^/yx 7 Y 
(3. t£fgSB««Jj£, SttSateg&T. isZXS/tttt 

x.7Yb^ &!8gmcoffi&}5 iV'To y y° Y em&Mz 
#><?>&$&^<rmtt£V/*lz\t*~?i?x. 7 Y b . A 

st ii^r ^ ^ x ? h i: . %:L>mz/foh\m., um^ * 

-Xi0f^*gE#»v-^r>yT*IS(C3iJD-rS. ®* 
~?—7T'v rmbtiih X o lzm\Zti& . L/£**-3TSK 
§ftg|52r . -e:fx£> j&%4 tL§ t i (f H T M L 3r f<7)±fi 

y *—•?•'/ btc bcoT&y v >y K =5: ^ VtlSli^ 
-y^'2 o 4 •cfifflS;ji4BWtfcitf^-£j£:r5 vf7 
t-AOI^ eyrufW < *^ . 34 A is < i t * s 

[0038] vm&Bmzam& &&/t$i\*xw3. 
^Y^-th^-yr-yy-muz^^xw^hm^z^ * 
h tmlv — yy > y y°wm*mm txmmzn 
h . ffii^Gu i com^ma-h c\btf-%mx'$>?> 5 . 

?l/y'7h^- K Wffi * W e b - b * 
istt. SMgu 1 -i y?—?*.— x&5H£tiX^&. 

z&Mxn. yuzJ-y b^-vmmiz^ tzbt&vi s 

a. MasterCard. S/t(i American 
Exp r e s s&£\ ffiffl$ixS ^ l^^'-y b^-F^O^ 

^rSrATJ-rsst^co^-c— ;uh'2 so^tiiis. W, 

2»7^-/lK2 5 2(i?l/y'7 ATJt" 
&Zk tf^mX'fo *) . *3«7-f-^K254 
RB&A^J-fS >! i: :5H"i6TJf> h . 7-f-;l/H2 5 0, 2 
5 2. fcJ;t/2 5 4tcA7 l J^tlst'[f|g$rg3M-ri>st46 
Hf7'5 y b (submit) tf'^2 6 4^ffi§ 

[00 39] H7t4, ^7-^ 7yhA^iiE^Uy7 
b*-FWffi^ffX#t-S7tft(7). HTML3-H^it 
HT'&S. HRt, <IfLfe<7)^-^T-yXSiS«Jg®fc 

'J71M2 6 2***4*l*. A^SP^-2 6 Otii. H 

1 —)VY 2 5 0 . 2 5 2. feiV'2 5 4 Sr^-fll^tffcO 
3— F. ^^V^7"5 -y h^y2 64W3-Ks&s-t 
(06) . ^<7)^lT-(±, l\yf^7"(eve 
n t i n g ) -tf^-F fcJ:VaiA6ji4^X^ UTF* 

xr-4yyi>7jk£tix&<o. tfys v Y#?y2 6 4% 
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mWr?& b , MWL r v e r i f y j jjfflftf H13*t* . & 
? U T hSfrfr 2 6 2 X*mfZtl& o MSSC r ver 
ifyj J±. ^rJWv Y%—Y ( V i s a. Ma s t 
e r C a r d , ti 4t/ Am erican Expres 
s ) CO*- H##3&9tWS*S"C* ft A» 1* a 
ft. 

[0 04 0] H8ii, f^BI^ffifflLtWebt-^* 
2 04tgffi^|,^ky>y h*-KflHI&]R»-r4fc 
@6(?)GU I fcPJtfc<9*£(fc*N&$'5-f T>h 

8~ai 6fcHLTJ£rmfce.ftj& 1 , Rwt«ft»tt, 

[004 1 ] — jftfc. j£3fi§P (-A^c r t fcnftf 
ftft) (4. rafWMi^KfitSiVDOM^r^x? 
hrnAr^I^itXMLgilt . bmm y^7i 

t£|WLTl£ffl-f-ft £ t tfX'^ h4 J <*>V$5Z.V : *V v Y 
b. V— X^—z/^CODTMFttzlZVffflfflbCDX^— 

>\s-t-v YX'fo&. mfflsn&atsivwmi. v-xy 
*x.*>Yemnbimmffix~fo&fz&, tmm±. h 

TML, XHTML, c HTML, XML, WML|*J 
X\ £ fc(iffit£>fl&tf) S GM L^tliMv-^ T -7 TZtfi 

{4 , fgWKT* -3 T 4 Off t V vfltil^- 7' *Jx. 7 Y £ £{4 

mmtJ i imWi£tiZ> s H y Y j-y'i?* 7 Y^lVCO 
«fcfttt4>*Lft . «SRfcov^«ft»-CffiWfci& tft 

ft. 

[0042] :;t, ifeSiSPfi. ^^^if^ntfsn-ft 

TVHXcDHfi^^^T, 2-5(7)^1, rmodejT- 
jB3RTS4ifctcfc«KS*utV^ SSlOt-K r Q b 
j e c t modejfll ^IWBA t flefflWCab ft . 

fciJf. XHTML^W^JScr ipt^fy^- 
7°y^\ i/c(±WML7''5^1r'c7)WML S c ript^f 

— A-fe >y hrtftt* jatt*U* 4 < » Ztit.it. t^'4 X 
«*t4oTi>M#SfLft. ob j e c t mode 

{4. ^<yf^y/fcJ;^? 'jr-f * ymmtx 

flW4ifc**T*4. *BJ«I«T'ffifflSaft^-. £4 
b *j 4t/x? y r-r >- ■y&^r-tf— Y LX oft 7 7 

>7-!f{4 r T-/7°K;l-7"5 , 7-^ ( u p 1 e v e 1 b r 
ows e r ) j bmttlt. ZcOWmcoyyWli, fi£ 



ygfficO-t^XcomH. 7dat^ XVvh\ fc4tM 
^yhW-W» 7771/^/1777f(i, — ffi. 

4 l 9*^HLll«tg^fiBX.7tf : Vs'4X(cl,A ) ^ft. 
[ 0 0 4 3 ] J£5figB{±. declarative m 
o d e j tfcW- h "Tft i £ ft . *BJM»-C« 

ffl§ilft:#-£\ declarative mo detl 
1^75^ r^vi/A^/^Hf ( down 1 
evel browser) j b Hf (4#U 4 -f > 

■fcfcfmt'Sr^. tfL^iO^J«7"5'7^'(4. 59r-5-0 
KSISSOWwW^ffi ( tth-h 3 Tg«fc 4 tfHtt ) £ 
"!f*f— h-Tft^\ fKtiODOM ( K^f 3-XyY*7'i? 

SM«S:O^S:ffifflL. SMIL (Synch 

ronized Multimedia Integr 
ation Language) 2. 03: h*<7). 

7/H-^f^ risiSB^it^BafatB (IWfcSfufc-? 
-^77711) tft(:flffl3*i5*^*S. 

t 7-f«4. AMWfc{4^aMt^MH§^^x 

[0 044] <1<1T\ ^^-K^ATJtcPV^Tf^L^r 

AJimcD? j -tYYi^-tfztsbltZ, b if-f77 

A^xt tt^ffiffl^-ft i t jWfcsfcffl-e* ft . 

fc. 7 ^ fti*«8fc4^JEf ft'tflB*^ 

«-f-ftB*fij^Mffli§ixft 0 fzbz.iL mecomxu. j- 

-f (ill C7 ^ -;l/H2 5 2t;^l/y'7 b*-h*#f- 
SrATJL. mzy 4-)VY25 0l,z7]si?>y Y$-Y<?) 

-thz.h#~z%h. arx"ik t ft 4 a tc . ^^ismj t a 

^^ft t . «#TeaK^KB^ty-^ a ysWl 
tt$^ft„ *HHiffl«-C«ffl§ixftJ*^. gtfe^7^-/P 
H mum i ^!mizi- hxw-y^^xrv-Ab^ 
WM^m-fj^mm-th^K-nmmn. r mu 1 1 i - 

mo d a 1 j btifl£tl&. 

[0 04 5] BJSHSSr#^ftt. HTML7-?7 

•y y°mm 3—y comwTr; ^fit^i.. h 7 1 ^ s tLfc h 

TML3-Hi:R«{C. l\<7)a- K(Cfc*Xat»2 7 0 
fc4t^'X^ 'JThgB^2 7 2j&^4*l4. S6fcH7fc 
^£ ix^ 3 - H 4: |al# C s 0 8 tc^S*ut ^ - N* fc , 

nffr ft »fptfo ^ 4 r $> mzm&ey^mz m-t ft fg* 

^'-i-StLft. %e7 4—)VY1 5 0. 2 52, *34t>'2 5 
4^ffiffi<7)A7j(4. ^^3— h'SP^2 8 0, 2 8 
2. fe4U c 2 8 4^4-5T$tJSflt7t(4^fi : S*lft 0 ft*3 
tn-Kg|5-^2 8 0Sr#Ef3L. t ZHSr?) U X 3 0 CO 
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b » -f^b TonCl ickj jMHttSfu -ix^'x 
? UTb##2 7 2-CBBft r t a 1 k j SPftftB-*-*** 

fctiisff*-*. zcommzx o , a^ti^ ^ -/i-f 2 5 

O T4^;K£iift-f-^<0?4 Tt=HBlftft S>*i/>^jSi8 

Tmu 1 t i mo d a 1 j tPyiftL, MWLCOXJJ&W; 

{tzbUi^&kTfO? U v^/n— 9) 
ft. 

[0046] #<«77 l J^-y 3 yt\ 
TT/^-ya "ytttf)?*??* «y?«»£teffl-$-ft 

Z t ft . 

[0047] WJK^SStMft £ , 3££f2. 
i£. N:fcffi. *fcli>vf:/y vh'Xm%bX'fo&tfZii 

■t ft wnmwm § *x ft t # , dtmf^, f 
safto ) *fs*i»-cii, r *^j tcia. igft^nfrr 

£»7^-^HtA*S*i4 t^S^xftATJt^JiS-r 
ftfflfffij&^ftft. v-?T-yTW!§<om<9*£3RgP£ 
*ttWUMW»2 9 0 (*^BiBHrrtt r r e c o j t L 
TitBJ'J) tett^fcK** 1 ***.** 1 , -^-3 ^2 

"f'&ib'^T^iSicS r g r a mm a r j fcilX r b i 
ndj S8fe6*E|jSS*tTV^. HRfc. Webt-^2 

3tffil4We b-9— v«2 0 2«*»fe««Stl.. 
-f r/bC^ya- F£ixft£\ Wi/Srfctffc 

immth Z £ ft . ftfcWtefcStfi. lOBttffi 

ffl-rft^fttCfg^— V^2 0 4 tCjl^itft „ g r a mm 
[0048] fgi^fut #1*. . s Jx X^-v . 4" 

*-^biznm*thimy--;<2 o 4 ft>&<z>s»t£5i 

SSttUtifcEfefc:. r e c offlffl!2 9 OCOflBt**. *f 

ft . «js$nfciai9BmTtt *ms&*??4T>h£3& 

ft LT H7 & fS^3& i §cT'f~ft t , r e c o ^i'' X ? F £ 

^§i&ftU SlS^f^f^ F t^je^-ft 7 4 -)VY 

zmmnnz . 2 8 2 & £ v 2 s 4 ^aucnf^ 



U &7-<—)VY2 5 54tC— ®^0r e c o 

h £S(tI?ft t - 7 4 — )VY 2 5 2 joit^ 5 4 %ix^ 

tiizmmmft>ti&« /n* 2 5 2«f 

IX 0 t-WLTfi. El 7 tBa LT±fETj£^ «> « f: H« 
«7fffiT\ MS r h a n d 1 e j f '9 4 7°*Wm 

[0049]— JKt-. T— 3rr?f->r 2 0 O&ilK^ 
-YTyhMv-^T-y 7°W1S fc 4t I^SOi & ffiffl^" ft 
*£\ SIC. ^-i.A.ttftS^MjiWt^^^-f- 

;p b- s6*^ s n.ft . s iiizmmmrnx'te , x^^?^ 

EB6sl«>f -;^i:'»ffiKco®ffico^^ffiMT'§ Sit 
2rJJIIf ^fLfcV^ r o nC 1 ickj ^t"cDMJEt"ft-4 
*!3tv-?T-y7°WfS<?}ffifflT'±<«^ftft 

K : Jx^i-v%b'<va^yV(Vfflffi£^~ffz!)bco r c 

nC 1 ickj 4K>hcr>mmi / zm%.ZfX&i>cr>TlZ%: 
Vii t ^lfS5^!tl(\ r o n S e 1 e c t j =5r£\ |BJ 
tBWtcd, ffiBcofltfflWSrGU I ^yhMt 
#ft 0 -HSt»®T(i, ^3UMOf^/*\ » 

ft w^corai&ti i 4 i ±»t <?d h 7f * svrir § 

*<OC»fc:*fflT2b4 . -3-— rtibt^tc^-— !f(9D*f 
K 2r it»-r ft 7' 5 ±THff ^ W7n/5A(:j:-3 
T , ^(OWtb =5rft 7 F i: ft £ 

[00 50 ] «-»4fB»Baii^- 'J *j6*, 8Mt 

-T-^'2 0 4frt>cv»«%:mi¥iS kV : /t.tz\t^^'& 
Wb-fZZb tco^T^-Tft . t&faTn-texeoll&rJi: 
co*l^r , b t HT -v 7° V <;P7" 9 ^ if h ?M 
^m%s tart ( ) iTttSy^yp^y^ 
7f't'»SfS< r e c o>B3R«0*^fc:, **Tfeft 

[00 5 1 ] 1vMC0M~ete, 7^ft-^77iJt- 

Sr«< P[3S £ b iz i . X^.<7)A7J^ 
$iJfP$itfto ^^Tr^^lfti. GUMOh, fcfc 

rri4i«»u *tjE^-ftiisft^Ht-. fe« 

7 'J -77 'J v--£/ a y^bcry^^Mmi^i- V 
3--*ftf7~7W$:ifr Lxmw\n-tz> z t &%wz 

ib, BMW-— -'^2 0 4 iitti? 5>f 7/ F 3 0{d. SM 
Sr Wit L T S: Mi~&fM ( «SWCH32S&fl- Lfc' < 

=5:^*^. S^tc. B»6*ff±S*i.*1ir(c«f»iailift*S5 
fi^SOfc ft (dictation) *3 i; t^ft&O 

( r ^--7y7^a7tyj tLTttt^ti. 
ft) . BB^«P3rff±»il*!6Wfc"1-4Af(-JT5r<, BiS 
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[00 5 2] — HJt^ffiTli. R e c o g*(4, ii'OJ: 

VJ.TCO 3 OCDfgf^- F £ KgiJ-f 5 CO r m o d e j 

TonRecoj ^f^yh^ifti^li rbindjf 

F j&qgjg $ ftT v V ^*§£-<D f'7t;l-h OSMft — F<4 
Tautomat icj <h ^S » 

[0053] 11411 ml$WM<F>fZtb<F> r a u t o m 

at i c j ^—Y<nmwz^~twmwrc*foh (mm^ 
-f, j^yhftkiimcmmcoiismiztm't&zktf 

h ) . B#PJM 28 111 Igfi-f-^' 204^283 

tin * M^-t h 4 0 tag* s fttz B#^t , tj i cmstt-r 

-A 2 0 4 36* 2 8 5 & mm L , 2 8 7 

[00 54] Re c og3ftO«*&-Mtt(4, BMWw* 
2 0 4 COlftf^^aHjffll-rS o Stt r i n i t i a 1 T i m 
eoutj 289iS, Bftc0M*ft2 8 3 
2 8 5 ^ TcOBfflSTca § o £ <Z>B#ia#* Hi 4 ^ , Btt 
*WihLJte?rfEi.S r o n S i 1 e n c e j 4 ^>h 
2 9 1 &Bmr—'*2 0 43&>t««S*l4 . BMWw* 
2 0 4 j&*B«rcS =SrV ^SS*^,o»t* k , r o n N o R 

[00 5 5] SH^ff^fcliffiOiTf-ri fcj&^S SflS 

cOMttfc Tbabb 1 eTimeoutj Jgf£2 9 5# 

JEEtT, SKri-f^yhje^^ffStiS. BMUT— A2 0 
4*5{S^i: LTW^^IILTV^*!^, fc 
fcK^fSf&75*§-£\ TonNoRecoj «142 9 3*-' 
MrSftS, Ttfc'L, ffiE^ffiJSOffl&T babble 
TimeoutjItt295 SiSS« 
X7-t*S^MA ! iS< , r on Timeoutj-f 
<y b 2 9 7^^T$fLS„ |Hl«fc, r max Time 
o u t j «|4 2 9 9 t> £ t fl^T*"?* 0 , £*L 
(4fgf§Offl& 2 8 3 A 1 fcl***^ 7 4" T > b 3 0 tCM^ 

t&b, r on Timeoutj^yb297*>lfi ; 

[0056])tiCfL s r e n d S i 1 e nc e j R143 
0 1 j: 19 fc£v*MBOT£jBAfc*£tt, BM^STTUfc 
£ t £ Bftc^-T t> O-Cfc 0 , im*t--> ^'204 1 ISMS' 

(4, BWSa*Bi^# T"$> 4 4» t" 3 3&»*W£^-4 fcfc 
cOSiflSJ/E (confidence measure) 



«k 9 i®^*&£{± TonNoRecoj Mtt293#'IS 

fis^x. wiaas^usv^ttiOfciKv^^a r onR 

e c o j JRf£3 0 3iJ il^aWS*3&%ffS*l«. 01 

4 (4, Tautomatic modej t', HJItfW^ 

stop () a-fotfmffZtiXi^^tZxk-f. 
[ 0 0 5 7 ] 01 5(4, lStt+>--A2 0 4c7) r s i n g 
1 e m o d e j M^^rt^MMX'hh . r a u t o 
matic modej LT ±IET^7tJKttfc i 

if a Ky b &mjmrn?i> o , ^-<d*§-s-(4I§i taw»#* 

T*§ftTV>l>„ fcrtfU £tf5tWF£— KTTfcL sto 
p () n-;b3 0 5j&>^^2 8 l±t^§iXTV^5. 
stop () r?-;l-3 0 5(4, J--Hf'(cj;.g. r P e n- 
u P j ^rt'O-f^btMJE-rS i<7)Klff 

^- b*f(4 , mmg^&m-tz t *», bh^w^t stop 

( ) =7-;b3 0 5<7)§IJWT(-$)S . -T^T^iSf^E- b 
t TinitialTimeoutjUt 
BH*2 8 9 fttS(fttB$*l*Vtl6^Ji r o n S i 1 e n c 
e j W O b 2 9 1 ^'Sltf Sttl» A*', C\<7MW^— b'T 
(4, IS^(4ff±§tt : 3rV\ H«(c, stop () a-)V 

305 ja«r^BiiT& =srv^ast= ± ^ § #ut r 0 

nNoRe coj -f-^>b2 9 3(4, l2MSr#jht^ 
V\ /tfiL, Tb ab b 1 eT i me o u t j JR142 9 

5 tfziZ rmaxTimeoutj Mtt2 9 9 {CRBStft 

[00 58] 01 6(4, ISit+)--^^2 0 4« r mu 1 t 
i P 1 e m o d e j mfEZ^tWMMTfo & . ffTMO 
4: 3 (C, iCDi&f^^E— b'(4 r open-mi croph 

onej <?5*^4«:(ipa>"^U3rrttffls*i*. hk 

(CI(7)t&^-bT'(4, BJK^J&s t o P { ) _3-;b 
3 0 5*^(tIX^I)5&\ ^:ti r babbleTim 
eoutj M42 9 5 tfzlt r rn a xT i m e o u t j 

Mtt2 9 9 c»awti5.*i^ieiai»*iBi«. £~c. m« 

r o n S i 1 e n c e j A Ky b 2 9 1 , TonRec 
o j 4<y b3 0 3, SJ^(4 r onNoRecoj -Y^ 
> b 2 9 3 <J0f^, Tbabb 1 eTimeoutj 
#^41^' TmaxT i me o u t j Wf^#ffl«0^-Y V(4 

[00 59] -jKfc, iOKFfr*- FT'(4, B»Stlfc 
#^J(C-3^T TonRecoj 4-^>-b30 3i£?&ft% 
tl, stop () n-;b3 0 5i}^ifM^>tlhtX'^ 
i>mZtL& . BBrrSsSrl^BRCfc 0 r o n S i 1 e n 
cej ^yb2 9 1^tf^!t^. i^^MK 
yb(4#S§tt^^', BiW4«ff^-*. 

[00 60 ] ltfj*<7)4 3 fc, 7 -;bb'ffltf0W3S#tt^ 

fife r e c o t/yi ^ b^'fBK?^, <ItL(C(4^< 
1 1 , ^ffi^ffiffl-TS IH^-A 2 0 4 «fif^£»t 
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ZbtfT^L ff M<7) i a (C , W J*T '- * Si . -3-- 
ATI Lfctf^fcWilff W^fufc.* h 'J - S y?7*- * ft 

mmmmztLfz^T-vz^tsz. t 4 . ffi^n 

tBKBTli. BJIWw*2 0 4 fc± -oTSttJR 6*1.4* 
J»r- * £\ ? 9 4 T > b fSTJt« W-fi L T ^ 4 =fc a 

? 5>f ry hm<Dmmz^f -9 <mmith^ 

&Zb j6*TS 4 o iftfciot. BMW- — ' ^204^ 

^^#jg*&fk3*u hmwww? 5 -f ry bfc it^ 

46. BMWw<2 0 4 (50X^-5 b" 'J r- 4 #± Df»mt' 
[006 1 ] »9w>"2 0 4£>&Bi»&gl£3ttK£ 

- b tRawt ^*vfc-r-^T a> 7 * b t^r-t 4 

t s 'If m±T7V dr—is a y%m<VlzMZWe b^—^ 
2 0 2 t;jS6ix4 „ fu3&7>l*l^£>, Webf-A2 0 

2 immteim% 3 - b * t i;/x^uni^ 
3 0 fc*gf£-f-4j&>\ nii-^-fxtiWe bt?- 

-a'2 0 2T*iZ%:<tsL?>mmy-—^2 0 4tcj;oTil 

It-A'2 0 4* ; We b-tf-A'2 0 2 fcftfcEHSfU. 
*\ 4 fc(4B»tfw{ 2 0 4 ifi9 i? 4 T > b 3 0 cr>— gp 
T"$>4 J: 3=5r§|jfeft*IEBH-4 fcfrCfcfcSru. m^SUfi 

04£>'We b^— A'2 0 2it(i?54 7yh30 fcffl 
&T£>4o 

[0062] msiz^tif^mmnum^Lx^^ 

UA S , r e c offlWKISs »W3r^f-?ft«if- 
A'2 0 4tlWtSM4£#>C'X 'J^E-b «mX75/x.7 
b (RAO) RAO^/jy 
* fi> x ? b fc-f 4 £ t Of lj jfifct ,f^^-7i- 
*fi»3r4 "FtHS**^ ifca& . 4 tfl^f^ *4* 
X £ tz\19 ?4 T y b Izft L xm% 4 fc 0^4 £ £: # 

rsftit-r**. s^ty^-b^iiiM-TVx^b 

{±, H^tlKRcOrecoSStjBttS-a-Sifcfc-r* 
4„ 

[0063] H9*J.fctf01 Oli. 3fsBJ*t»Tll«SSfL 

fltefcHTMLi: LT^-r@T"^4o H^6BJ6j6^J; 

#3 0 2^illtl^ o ££W4-7-^T'yTWl§<?) 
ffi<7)jffi;5figP. -ffthibb a r g e i n£i:'CDJRttft#ir 

rn>-7°h$ijffli3 o 3#**. fc*fu ^pmmn. m 

9fcj;!>'il0 WtWfflHifitiil^r o Xft hit 
4. ££-C{4Tn-fex(i, ^co^-^bftWSU ft 



4, X^'J7Mt TcheckFi 1 ledj Cio 
-C^*W«0f|l§*i4 o t^t. rJtffili, 

-r- 9 is X Xfmmt 4 n* 2 0 4 1 

ffiffi§tl4. Wtti^-^2 0 4*>ii>S(t]Rofc 

a£W±, ryhc7)7'f-^KtM3i#{t6tL4 

(=WBiW»'C»4CRW^9^ir2 12). 

[0064] — jKtc^^fflTT y ^r— ^ h ytzm^co 
mmm± . ^pmmzti^fr ~>tzt% ^-^zft 

•T4fl^T"$>4. iS^CT^ft-r^Tr'J ^~ 
i'gym. r on NoRecoj (i, ilLTV^^ 
;fc&^HftC*iSS*Ut7-f-/l'Ktnu 1 ltt£A 

W-RcomMl&MTlZ, TonNoRecoj 305 fim 
iTmumblej ft ■K^ffl'^4fcJd3Sff t . IS^ft 

10-9--^' 2 0 4 tcteil Lfcf*. ^a^Bi^^^x b 
*^tMO^yXfA3 0 7 (H5) ^ffifflLT. 
ff^fcSEHlSiii. iriS-f-A^ 0 4{i"Tllix h y-A 

%m&g?i7'7 i yy £ 2 1 2tCML. ^C£tl^'fl:fg8 0 
WEiM§*LTi-Hf^M<£i:fc^4. |a|«fc, 

«> . t § tcfs^^f-^ ^'2 0 4iz£~?x ^tux b y 

-Mz^M^ixh. 

[00 6 5 ] ZCOWCli., MWL r we 1 c ome j 
U")aWi7oy7btB4U«, WSl r c h e 
c k F i 1 ledj j&*:3-— iFfcfcf LT#7 —)V HtCO 

f 4 i>i7)T"a5-5T. ititcii r con fi rma tio 

X&ttVltZ: < X 9 y T b SP^ 3 0 2 feMI^S tl4 Z 
[00 66 ] *^BJ^fl!l<7)fil«i: LT . 4 ^ >f T» 

A T y b ^ W X ) T^T^TSg^ V- ^ T y TWlIti , 

-enm<7)^7^r>'b7 : vswxt>fffg-r4We b^- 

t^VMA^yhcDdh^Kth lotft&ti. iti 
(iJffC, We b-F—ATTU^r— ^ 3 ycOfiS^-gE^ft 

4 < « & atr £ b ^"C# 4 fiOT*f UT* 4 . H 8 i V 
H9,Hl0t,B» r handlej ft^MLT-^J 

[00 67 ] H9tC(i^§#lTV^V^\ V-^T yT 
fiCli, DTMF (Dual Tone Modul 
ated Frequency) UfflU JzltVfffimWm 

1 *j x. 9Yb\^. nmmmzytf- b-r 4£ 
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tbCD&MM&Z L> iZ 20&4 „ DTMFIir e c o SUftP 

a. nmxmmtzmt^. 

[00681 01 1 H £im 1 214. tWfflWt- 

f k# jS^rffico v- ? r •/ r mm £ ^-tar* 4 . £ 

fiOSSSBBJi-Cte, ^-"f J4fif$B#A7J § S41S 
§ t § izffl h bxnwm: 'tfoZk #T S 4 . S v «| 
■ishJf, ^■f^.l±3--lT<0iStBIBftS-a-S*», -f-at' 

mx-fohi)K ^-mmmzm>&ztifcmL(vffimzm 

&t& i fctf'CS 4 . r ±»aiTb^- (mixed 

initiative) j CO— MT*£>4 0 — MZZCO 
WZM-PyMafc (dialog interaction) 

^"C# 4 . a-fi>'7o y7 F #ufeJSLh«flH8 

[0 0 6 9] HI lfcJ^'Hl 2<59#Tm. rdo_f 
ieldjt LT^SftJtlfcffifc:. r g_ c a r d 

typesj s r g card numj „ joiV 

r g_e xp i r y_d a t e j tB«Eft(t4>*UtflWH 
#*#S;fx4. <r<7)MT(4, r o nRe c o j T^§ii4 
J: 3 teBaSfufc^fcSWRS k , WSW^r/^lf 

2 1 2^«fS8o^^®tro-?^w^r-^i3j;u : '*ffi 

r do_f ieldj *ttfflT*/£»f5JftS*B»9^- 
2 0 4 ti*f= L , W^r- * A»6 «0B»$*ufcffiBi 

^t±-r^.-cco7 -f -71- Ht=w-rs«frHmft»t* i * 

-#tfWlC r h a n d 1 e j 36 s , P¥t/ffi£fl4# 1 £j'iJ4li 
fT§:?x4. S^iHiiftff, 0 4j&»&JK»L 

fcb i ndSiU^oT. 7 >f — ;k F fcHBSft 

tt£>;fx4. H5fc3j*3:h.4J:3fc, BiHf— ^2 0 4li 
**#*T§IE 3 0 9 *£tf£ t **T*§ 4 o 
[0 0 7 0] H7. H8, H9. H10. HI 1. i3=fc 
l/Hl 2T(4. ^IttCfEGI-^We biH71/-A7- 

#8£T4C:i:t\ M*S77'J^-y 3 yra (y^fA 
±**Jj:tf±iWK&£) afcliaift^^'J-r-f (mo 

d a 1 i t y ) Pal ( GU I We b<— X, #^ifffl. 



4. 

[00 7 1 ] ±IEWJ: 0 1» r-7TV^y7'7Hf{4. 
BWfeRfcWOSTifcftfc. ±IB<O0ttW» r han 

d i e j sm/m-tz. k%i?ttM*%:'jmmm*mft-t 

X? V 7f -f - £ 4 . 

HffiE<50H]Sil^fcJ:V#ilAcO2. 1. 2 1* 3 & KiE^ 
4 J: a r b i n d j ®«(4irit*SII£»frL-Cffi£ 
ffl^tl-^OT&oT, r b i nd j S*{4 r r e c 
oj K*«lf735**fcfcfrF38rca4. 
[ 0 0 7 2 ] X ? >J 7f * >yi±^mT'$) 4 ^ti 

56BBOffi«Sai9^4feliJII«'CJi» r bindjIS 
iimimm ( r r e c o j fc B») T»oT , ffitoJ; 0 
SS^nnr ^ ^fi^Tfe 0 . ^nWlziix? U7f 

[0073] x^yrt-fy^fifflL^*, ±tz\± 

WjknM.T'Ci&t&fflmcDWzte. ^y^We b^ 

77 U ^r- a y|sJ«£ «f-r4 i t , *J J: ^-^'^ 
?5>f7yhfAM x fcjgftf-4 i t i o t co^.54jt 
T'^4tOtfe4. *5fiBBf5i*OJBWfc:J:"5, 7d^7 
Vi4. ^-A£7)5^>-H h U 7 75rLC^-y^7'y 

[0074] m&commmTte . r b i n a j w« 

U . SaBSHSrftrii* ?t<4W e b ^(7) 7 ^ -;y F 
fcWW %X htltbCOmH r Ta r ge tE 1 erne n 
tj feit/' TTargetAttr ibutej <7)^Sr 

*T4 . mcommwmxii, r b i n d j mmi± r t a 

rgetMethodj i>^A,X'lo *) . ZtLtf^i/x. 
9 F^V >y FDfttitffltc:iIJ[I§a4. r TargetM 

e t h o d j (ofltffltj ilflMBi, 'J7°t- -f 

mmhX. *-7 : Jx.7Y roB J ljcorxj ^y- y K 
SrD?tJ f m^-i£* s T#4. 

<bind TargetElement = "0BJ1" TargetMethod = "X"... 

> 

[00 7 5] *BM|H*-c-^§ft4MiHTML/XHT 
ML-YO F atfcfcfi! 3 «. <0T2b & 36* . S^»C0Sffi* 
"C"S>tL{i\ W3C Document Object 
Model Level 2j^liLevel 3>f< 
Vf^/P, ECMACommon langua 
ge Infrastructure (CL I ) 
F^X/K Java OiifSfl) 70/7 5 >-^"Wi§^ 
'^/'htf'l', W3C Synchronous Mu 
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ltimedia IntegrationLangu 
age (SMIL) , ±V®H«?^W3C X 

CDT143rV\ ft!2<7H O-r-f vyW$i^-, <b i nd> 

[0 0 76] HI 7fc4t>'01 8<4, ?7-fT>"b. flr 
#cWtC{4 downlevel browser JiTHfir 

«BW 4 . r bindjMIi; oSffltSS t J5E t T , 

4 t . -T com^-t-^X cr>X'7V7'T4> 7°% L TW e 
b+r— A2 0 2 

[0077] — HI 7i>4t/Bl 8<7)V-?T>y 
7lC(i -T~jr-b?i'9>3 5 0, f^y 3 y3 
52. £4tta_— if-^y^— 7i- X-fe^^a>-3 5 
4. 3 5 6. 5 8tfi?&tL& < , -^^->3>3 5 

4i4. jl— r^a^4«*ftfcBW-*HRPB#«-3&»fe 

n-£j£&„ JMfcWfcS*.!*. -fe^^a >3 5 614. ? 

4, t?y 3 y3 58l±, ttAftOIIS tW^- 4 SIMS 
S£rSfmg>„ -fe^a >'3 6 OiifrU^ •y-fe-^> 
rsMEXj *ffiflW"4lifM«r-fe^ ^ a 

[0078] MM^i 3 fc, ->&ffl)toz.<nwston r b i 

n d j SiCI^7'yx 7 b * V -y b'Uf ft L#^£;fl. 
Z.td$3 6 1 T" r w e 1 c o m e j ^7'yi ? hcD r s 
tartj^V7 FtfUfr^ifafci: It, r we l c o 
me j 7Dy7h^SWI 1 ;ttJ;-5t, HI 7fc4 

[0079] iRtli- Tt4. 362t r askedjt 
TVx? bO T s t a r t j JV-y F£Sgfft& <I i: t£ 
i-?T, r Do you want coke, cof 
fee or orange j u i c e ?j t fz^fa 
?>tl& . &VVC\ 3 6 3T r reco_dr i nkj 5f 
TVx ? r s t a r t j *V -y Y&Wtf&tZ. t [Z 

[0080] ^:t^y3>3 5 4«V-^T^7°* i H 
ffSft. KMl^*2 0 4#ffiJlW-4:fcifc&*, Xpat 
hXf-h^ L /drink typesj tc4 
^Xm^flh, rf)Mt11 W3C«X P athfiS 
fcffiJHLT^***, 34Hff<9SEtl5#T2fc*tff, W3C 
XMLRg^affi (XQL) &#tfA^fx^H/t$^-g>i 

t vMfw>awmffit= s Tis±*/£{f s £ t 

ft«ifc*ai!W4itfc«KSnfc^. r bindj 



*tKo)tBi«ia^)WHi»* 1 o 4 o t*sv**&, 

3 6 6t7D>7b*7yx?b r re promptj 
#\ inV^T 3 6 8t7ny7h^7i'*x? b r a s k j 
&mf£tl. 3 7 OTiSli^-7'^x^ b r r e c o_d 
r i nkj iffim$tl&. RSflfcBWS***, WSfi 
**1 0 J; 9 t^C#^ r c o f f e e j C30*|^r. 3 7 2T 

•7j-)vy r d r i n k j (csmtf&zH&wn o at ^ 

ft. iJCV^i— Tf4. 3 7 4T, 7oy71^7'y'i7 
b r C ream_sugarj t j; o T :? 'J —A 4 fc(4 

3 7 6 T\ t?y 3 y3 5 6 OfSii* f isx. ■? b 
r reco cream sugarj y^Dfl/ttJ § ft 

-g. . iixt ts^scoisftfi^ i o 4 o 

tij^'c o f f e ef&l^-g-. 3 7 8T"Bffid rink 
7 — ;P F im 0 ST ^ixS . 3 8 0 t7n y7 b 

b r c o n f i rmj SStffL. ^Ofg-fe^a 
>-3 5 8«3 8 2t12i^7yj: ^ b r r e c o_y e 
s n o j SrUf^'tB-Ti t fci-oT . ISfiMScOWfg^ 
i . JL—^* 1 10J:0*I V^lflT" r y e s j 
h^-iSh. 3 84t'7ny7f^7yj:?l rtha 
n k s j JtP'ffitSfU ^-^f*3 8 6tfS^f7S >y b 
i#tfc(431V^ W r noj 

4 ^{4f2ilie*<735iftffi* i 1 0 4 0 /Jn$ V v»&, 3 9 0 
t7ny7ht7yi ^ b r r e t r y j j^'H^fSiX. 

t> d— &3 9 2Xy°uyyv^y' t J.x.9b r a s k j # 
§#1T . 3 9 4TlSfi^-y^x ^ b r r e c o_d 
r i nk j ^Wffi§ix4o 

[ 0 0 8 1 ] mai^M* 1 ^*'* 1 ^ 4 o iz^ rbindj 

S*(4. t?y 3 y3 54. 3 5 6. £tz&3 58t'^ 
-T4 a (c^ V v b'coaitDf tf} t & "Tlgfc-f 5 . WMTft 
ixif . ^Sfifc«ac?D«WWaT 4 ; k i>T 

[0082] fffi<5on^^-c(4. ^ y -/ ^mzmt-fc 

{45ll: , JXb^^^t><7)t>^4. itL{4, r argJ 
7T-y~? 

<bind TargetElement = "OBJ" TargetMethod = 

"F"><arg>X</arg><arg>Y</argX/bind> 

{4. TOBJ. F (X, Y) j tWmX'hhtK fohX^ 

(4 roBJ j tiK J^t-f&tz^m. r Xj fc4V 

r Yj s-ffiffl-rs^y >y f r f j ^^r-rs^-y^'x^ b 

[0083] rbindj gjfti4. b i n dff^{:(?) 
^^ylfflt*l.^fftl. reventjltt^ 

<bind event = "onNoReco" = TargetElement = 
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"promptl" TargetMethod = "start"/> 

ronNoRecoj-f^yh^W^i&S 
flef S i>COTfo 0 , t/y'i ? h TpromptljCO 
r s t a r t j * V >y KjWtfftSftS H k fc&S . EI 
8 tcW LTJJEcoWCSMH Ltz =fc d r b i n d j g 

r R e c o j LTffifflf hm^hm 

^^Wz^htzMZ^ rbindjff(?)f7t/l/b 
JKttti r o nRe c o j 

[0084] »®Mmt LXcr> r b i n d j WMli. H 

r b i ndj mmit. r^-tx^m 

S r s t a t u s j M'S^r-fi^S . r onErrorj 
-{'•O'b rbindjgf£?)i« 
4^yb* s r s t a t u s j M14£Wl>-&*§-£\ Ztlt> 

[0085] 8ttlfi&3&ft£?-x y ?-TS £ 2: fcJWi 

>y i fc j6*TS & . ftfleWfc^ilf. Uestj 
JRttfc ±tf fvaluej *rx«M7J£ . ffittK^jL.* 
Vr-cO/V-by— F£^t rhostj y°V 5 x 4 

^tfiafcatsB-rtii:*^**. ^t^ii hi 7, 

a >3 5 Wc o f f e e t^tlt 

46. Ltztf^X, -?—?T-yy° r hos t ()/get 

dr i nk/d r i nk= r coffeej j j; -? 

XfBMZtl&XotZ. *l91s*iV?> 5 6 £«ftfttWfcft 
(7)7 7 ^'ii. d r i n k 7 —71/ b# s r c o f f e e j 

[0 0 86] ttz, rbindjg||^f-K20 

4 f ±»J ST fcjgfflTOSrC }ft'& < . t £ If , 75 
4 tv b^vsi-f x±.xWn^<nry° u v--y 3 y*»6<?5 

^ -y-fe — ^7'^i ? b (^WmMXte r s me x j X 

7aztixu&) iz&mm^nx'fo&ztiz&mztifz 

\\ Ml 7 tiki/Ml 8cr>fflTl£. yvATyVf^ 
X±.xm?r*cr>mmTy° 'Jy-ya SOP"* fcStffl Ute fc 

r b i n d j .X -y-fe— V r/C a 1 1 _c o n 

n e c t e dj i^^ifM^tltzt^ liZ^ r we lcom 
e j y"ayy°h%:m?f£fzim&L. r reC o„dr 

i nk j ^■y^^^h^mf-thzbiz^'yxmm^ 
m-z . 204 ^stt«5 smsm^ nm 

4> J4 - Fir l«7n — £■ fflS^f 4 fcft (cM 



§t« t isi« t . v-? r -y ritmmth z t * . 

#3§A<7)r e c oflii, ZCDMifcZMfi-f lizbb^y' 

7°nyt-f 4 Tinnertextj SrffifflfS C t± 

WH&7 r 4 /WCx^f X >yfe-iy*Srig«-rS Z t & 

>'^"t|5l«t-ri.-i:* i 'T''§-?.» Jtfcitf. -i^yf^ 
y?M r on Rece ivedj Z&tJiZ t if^HtX'h 

o , ^•yt-v'v-x ryh 

T)M XlX'mft^TTV y--y3>-) i ! 77^fC 

[0 087] LfztP-oX Tsmexj ±tzii^ >y -fe— Jy 
^-7"iy'x^ *H3*H«T-ifrtS-7-^T-y7^^" 

Y«)SS«^«jlfttCffifflt-S i t ^'T*# TTY» 

t J: a tffifflsn.. ^ •y^-^mm^tixm^coy 4 

-^HfcMOS-C^*^, 4^{±HUM^ r reC 
oj s r grammarj , 4?tii; r b i ndj ^rt'C 7 ) 

•y-b-£y 4 ^{i r sme xj t7'y'i7 ^tOI,Tll 

[0 088] rbindjSfll r f o r j JRffiS:^- 

Jf , tlTcO J; 3 ^rv— 9T-y y° 
<bind for = "prompt 1" event = "onCoraplete" 
targetElement = "prompt 2" = targetMethod = 
"start" /> 

fi, ^y"i>'x.7 h prompt lj tf'A <y h r o 
nCompletej SrMft Lfz k # ^"7"i/x 9 V 
Tprompt 2 j COs t a r t^V -y h'^BfVtB 

■f. 

[0 089] SJSH5Sr#^«» Webt-A'2 0 

2i±-if-y^M£07°7r^ yiM^r- -if u y^y-ziy^ 
fcii* ^*j.-;i^3 2 o i-ttsz. k ■& ( ^fc ilf , 

Microsoft Corporation tCiSA 

SP4fcliASP + , J S P 5- k' ) . -y <fl!j7°5 ^"-f 
V^iy^-ykS 2 0(4. 7 y-iTy MJ?Jv— ^ T -y 7°. 
^•^lA"(CWe b-^— yS - 2 0 2 ^T^-teX^?)? T> 
h- <D9 A y° C^fiTfir* ? T -y 7T" S i i> , KjW 
t=**T Sit So 7 y A T y h /*r-f s'tfo 
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e bHf— A2 0 2l<zm%Zillt)\ ttliWe b^— A 
Ltz? =7 A T> MU^A- ( #77'J ^-i^g VTffiffl 

•ciirecofc^/oy/h mm ) zmm-th z t 

[0 0 90] ??4T>ha°?—?T'y7'>£mt/}fc£j& 
-t&ZtizmZ-X, 08, H9, iJiVBl Oeov-? 
T •/ 7°#l£flM LT06 izM^Ltz b A— ¥ffi 

4 Tn/tya-zWi. 77 

- ? Klcfl* vC , ^7^(7yMl7-^77 7°& i. t/A 
7 U7°h£, 3:Tf~?}V^—y}V<7MjiC0i/ 
i- V *VW03lz±iSfr& . afar >f 7n^ 
ti . HS6#tf>--Xfc-£ 3 J: a 5 4 T y t-fflh?-? 
T -/ 7° Sr »T 4 fcftW^ 7 * - ^ * * tr £ t **T § 

5-f7yf ilJv— ? T >y 77 /*U7h A* ¥<7>?4 TCD ? 

Z b tfiX'% 4 . t-^'175 yty'a-^3 2 Ot 

4. 

[009 1 ] J2Lh, *«W(cov^T, »4 U»l 

{£. #fS^»»fcJ:tX«gH£3M^r 4 

[0092] ff#§A 
1 titfefc 

-xbtfeS. ??'f3, HTML, XHTML, c HTM 
L, SMIL, WML, sSrifOi: a ^rffiKcOS GML# 

^7"fi, Vi^y h >'Jt[R e d m o n dtDM i c r o s o 
ft Corporatio ni^lfJlSflt i < 
m$>tVfcXVvV"Ct>&. SAP I 5. 0t|^»"C* 
4, S*, -fOK JMf, tq^t^, MOffl 

L T Ji3r Jo 5&r v> . ^SWTIi*^^ it/DTMF 

B»fc-5 v ^Tffbs t t ^ s , Rao * yz^mmm 



[00 93 ] *gH«WCifttfe±*reiKi, JJATOSI 9 
<prom P t. . . > WjS^KWj&fc.fct^Oi'T 

<r e c o . . . > mmmmm^^mzm^wn 

<grammar. . . > XJlXmV V-XZmfet 

<bind. . . > wmsi&9mti>i*& 

<dtmf . . . > DTMF %mi&i3i.U"fflm~t&M 

t \ 

[0094] 2 Reco 

Re c oSill -BJtg^rjL— TA7Ji3j;VA7J*SS^^ 

X , ± : SrS*{i<g r amma r>±jj;t5<b i n d> 
T"^ 0 , BMSKOTnvtT- ^ * t8^-fS7t«>cD 'J V- 

[00 9 5] Recogtll S t a r t £ It/'S t o 

SM I LrJ-?yh'£ffifflLTSM I LtMSfflW3r^5 

tc#ft-f"§<IhT-, 7*^>l/^7'7'/f i-f^iyh, 
7>Jy°b <f ^ ) •CliamWfcSS 

IK'IS i »Re c oS*Sr|5]B# 

[0096] Recoil HffCS Sfg^^^y 

i-, tgcD^- h\ t"=5r*>^> Tautomaticj , 
r s i n g 1 e j , i7"c<± r mu ltiplej 2rSffl 

[00 97] 2. 1 Reco^yr/V 
R e c og|gt(±, 1 04fcl4«BR<?XjtffitS±tfffiiai 
iKX'b i n dg*C0-fe >y btf^&tl, Ztl^MMm^ 

l , mizmmtz at»**wic^- ^*tc3 1 --t- 

4. 

[0098] r'yrVW?')^'!!^ R e c o#'ffl 

[00 99] 2. 1. 1 <grammar>gi 
gramiarglll, s r cltt^fffltS-f y?-^ 
y 4 tz l±#H5arA if 6 * *B5&*- 4 O tcffiffl $ 

^^{i-r^fx b^7it^^ty?47T'fc'5 
•C±V\ fldR»grammarfi**ifi^-Sii:36* , r 
#4. Si£^grammarM«?n^ 35; 
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[ 0 1 0 0 ] : 

■ 1 a n g I D : fflEHKTfc*. ifOaBW&xyy 

77M±xml : 1 anggiti^. fcfciHf, 1 a 
ngID= r en -usj (i*|lffifl£ ;«Itt 
tt» 1 a ng I Dj&nfc&UR I teflB&5ih.T^3r^*& 

[0101] 1 a n g I D#'^@EfrT"f£5g£;fxS*§ 
1 a n g I D(S«ffifirffiH*^M5feJHfitc^a 

l§ i d tfXmmmftX'mfcZ tl& )»«tgramma 

(grammar src="FromCity. xml" /> 
or 

(grammar) 

<rule toplevel="active"> 
<p>from </p> 

(ruleref na»e="ci ties" /> 
</rule> 

<rule name="cities"> 

<1> 

<p> jr>?V<s,& </p) 
<P> i'Thf </p> 
<p> n>H> </p> 
</l> 
</rule> 
</graamar> 

[0102] s r cmmm^ma ±tM 

<sml conf idence="40"> 

<travel text=">-T WP^jtfX V y^iT#/cW'> 
<origin_city conf idence="45"> y71/l/ 

</origin_city> 

<dest_city conf idence="35"> ^Xly 

</dest_city> 

</travel> 
</sml> 

[0105] -%mx'coi$mi±. MM-?~7T-yy°Wm£ 

§^-CV^Jt4t>, SMLF^a^ylKbind^ 
*«iXPath!B££ffifflUTlWHS#tS. ffl 

^X'foh (BM®mt%&Mj2 ! frX'$>g>) tzih. Zti 



as. 

[0103] 2. 1. 2 <bind >S» 
b i ndB*(4BfMBS^60it*^— S^tKr^-iOT 

[0104] bin diiti o-ra»s*iSBWSft 

(SML) ^ttfXM L R > 1 -£ 

WifHl^#*aS . s M Lliftf«BiSjRR*i.< 
tf£fc£>T#S (NftffiBmaKatt£-£H8) . r*^ 

ML K*A*Vb«W*J2Trc=3jr$\ 



[0106] JRtt : 

•targetElement: SML*»40v 
a 1 uea^f^y^DattWBR (W3C S 
MIL 2 . 0«*i^rt|H|«) . 

■ targetAttribute : ffiggMR. SML 
H«va 1 u enyT-^y^f'JD^TfeaS^— *fv 
bS»«S14 ( SM I L 2. OWattribute 
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-n\ f7t;l/fOfl± r va 1 u e j „ 
•test: fflM. WmBdm*)*SXt>tL&&R- 
XMt Pattern (W3C XML DO 

Mft&em&kmm fy^ivh^mtt ru 

e . 

•value: MM, ?-?"J hKRtWJSTfefL* 

sawsaK^fA^^h^^atJBsr*. xpath 

(W3C XML DOM^^til) S^50. 

[0 1 0 7] ifESML^*)*^- lil 

TOr e c ofl^i, o r i g i n c i t yfcit^d 

e s t_c i t y |*I<7)ffl&. $—*fv Y^—VW^it x 
t B o xO r i g i n£ At/t xtBoxDest 

<inpnt name="txtBoxOrigin" type="text"/> 
<input name="txtBoiDest" type-'text" /) 



(= , SfeftftS tn^t 3&r=TirC» bin diJjfr^ 
Hfj^frt LT. d e s t_c i t y^S^WIWft^ 



<reco id "travel") 

{grammar src="./city.xml" /> 
<bind targetElement "txtBoxOrigin" 

value="//origin.city" /> 
<bind targetElement "tstBoxDest" 

value="//dest city" /> 

</reco> 

[0 108] Mb i ndll J^lT^Mfc^SfxS i r> 

<b i nd targetE 1 ement=" txtBoxDest " 
value="//dest_city" 

test="/sml/dest_city(@conf idence $gt$ 40]" 

/> 

[ 0 1 0 9 ] b i n dgfft{4, down level &?c 

5 ^"f ht^reco D OM^7*y'x ? h 
fficOf^MaSr nrfgt-r &tmbtz. o n R e c o A <y h 

^yv^^mmth. 

[0110] 2 . 2 IffiiJil/rnA'f ^ 
[ 0 1 1 1 ] 2 . 2 . 1 «tt 

tMTcORe c o«Iffli, ¥4 Tu?9—y (dial 

o g turn) (=w tTWWKHagaisaiflw* «fc 

• initialTimeout: ffSSHRo igUISM 

h7t-AKo n S i 1 e n c e YifiW&^tl 

6 (2. 4. 2£#M|) . »5&&;h.arv*»£\ W^7°9 



•babbleTimeout: ffigSfL 
BtlSM^o automat i c& itXs ing 1 e ^E— K 
"C^ia^tJSfflS^S. r mu 1 t i p 1 e j ^E— H 

#{4. *rxm^**^§iX'l»3i/ 1 4^{4ffiO-f ^.V t 
ICSIBSa^. X7-i>mUd 5 
a^fc^yr-j&^t&fi*. SHgliE 

SV^|gO^^:i:"(±, status 3- F 13 (2. 
4 . 4 £#HB) t'onNoRecd Ky h t htl 

BMB6K«0X5— -CftinTiMfeWSEV^*. o 
nT i me o u t b#*&t ^ft-l. „ fgj££:fX&V> 

#3-14. t^75 -y h 7 *-AJ4-r7 h- fc LXftm 

■maxTimeout : ffiffiHtfL 
*l£@JiSf:, /WEi^TonTimeout-f 
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£ MiBMB8&<01tmtzj£ l&t, <OX'fo 4 . r m u l t 

i P 1 e j HT'<7) r e c o <7)*§-£\ b a b b 1 e T 

ime outzmm-t&m^tmmiz^ z<wmmz. 

IB§fL-g) „ maxTimeou t Jffl4(i. i n i t i a 
ITimeoutJjiVbabb 1 eTimeoutcD 

i 9 v ^ * id^p t < * »mtr * ^ * i fc t 

■ endSi 1 ence ; HMMtR* automat i 

COB^falf^ r e c o CD^^li automatic I^7f.<7) 

■reject: ffiSM. BW§5«L&V>«rC*-> 
t\ £ it £ THIS t77-y b7t- AJi r n 0 rec 

• server : ffiEiif/L m^P7°7 -y b 7 ;r — Ac7>U 

r i ( ^^-f y^-ru ^tmmry > y b7t-A* { * 

erver = protocol ://yourspee 

c hp 1 at f o rmtW. 77 U v~— a 

f±. *tf =fc :HC U R I ifc^J^BS^^J £51 

protocol ://yourspeechplatform?bargeinEnergyThresho 
ld=0. 5 

■ 1 a n g I D : ffigjMfK* fc'<DWfS#jSxy^>£ffi 

ml : lan gS8tl£5 . tihZ.lL 1 a n g I D = 
r e n-u s j fcfc, #HBSI*jiC*\ i<50«tt{±. g r 

a mm a rg#g£ 1 a n g I D^f^g£rfx£r^*§-^cPfi 

oTtflt'*!. ( 2 . 1 . 1 £#H3) . 
•mode: ffig«0?. SI^rSBWE-KSS^* 

ifc^L fflJ£3*u5:V*##li, f'7^btU r au 

tomat i cj -t— F £ h h „ 
[0 112] 2. 2. 2 7n;tf^ 

IS** 1 * * ft S ( T -y 7 5 ^if \>z i o T 1W- 
b Sifts) . 

•recoResult: StlX9*ffl, 2.1. 2tfS 

tUJ:^:, E*v-^T>y rwlS (sml) £-£tp 

XML DOMy-Wy'i? M^fcfiM*$#t4B!M 
cofeH. MiaaHS*i=Srvv*^, 7onf ^(±nu l l 

■ text : BflR9/»5&». MJtSft^mStOx^fX 



coResultt SM LfgflM 0 T'^BtifffigfRcD 

§fts. ~v—9T«*-r9y<m8ti&x. 

3£#^!l{± Tsmexj ^ >y -fe— v^TVx ? Fj& j £> 

■ status : IfWtl, |gfiT5 -y b 7 A£> 

^110, ± Jt(43feBc«li- 1 *»6 - 4 (Starts y 
7K (-b7ya72. 3. 1) fcMAct i vate 
.X y >y F (-fe7y372. 3. 4) T"<7) nTf^Wf-^Jg 
8 § ft£ h tJ •? ) . =Sr 6 tffcBMSaM ^71 <7)SK 0 
BSfcRS&SiftSttJIM- 11K-15 ( 2 . 4 

as) *c*ft. 

[0113] 2 . 3 ^yVx ? y >y F 
Re c oCOffiW&it&it/g r amma r WSlJ-f fcti ,. R 
e c o(ODOM^7yi7h(:i>^^^y7 KSrft 

t , T >y 7K/1/77 ^"tf ^' Re c 7y"i y* b 
SHB&ti J: i h , 3Wf fwoBMttffiOiWi 

t, 'Sr^VWi^^&SaeSS'J (777KII/75 

[0114] 2. 3. 1 Start 
[ 0 1 1 5 ] mX : 

Object. Start () 
%L 

mtl : iO-Xy-y ^-ifncOs t at u srj— h'S- 

(fire)ti. WSrPWteii, S^SA^V^ ( r e 
co s t a t u s =- 1 ) , ~$HW&'3>.>M )VWUh 

nmizm « «. z t <^>h &xm<?>v- h n 

URItf#4Li^ (reco status=- 
2 ) . t^(±W^7°5 -y h7=r— ( r e c o 

status=-3) tfi?&tl&. 
[0 1 1 6] 2. 3. 2 Stop 

a-)VX-hh, Rec o*yV*9 hi±^pC0»^W 

±l. immwte^m'>K^izm~t&mm&m , 
$mfri?±Ltz9$&£Tm-f. Rec o^'ffiM-rs-r^ 
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iW±-t&CO-Q^ automatic ^r— 

[01171KR: 

Object. Stop () 

[0118] 2. 3. 3 Cancel 

[ 0 1 1 9 ] fltSC : 
Object. Cancel () 

Wh: 

[0120] 2. 3.4 Activate 
Act ivate^y-yFIl SJRSfrfc^ (CFG) 

^ftstsaaift . &mm. r s t a r t e 

[ 0 1 2 1 ] fit* : 

Object. Activate (strName) ; 



• strName : *SSWfc$*l..&&M<0£1& 

[012212.3.5 Deactivate 
[ 0 1 2 3 ] flfcfc : 

Object. Deactivate (strNam 
e) ; 

■ s t rName : ^tMfc^SSOWO^Hrh 
Wh : 

[012412.4 Re c o-fOh 
Re co DOMt/xx? M4J^T<7>-f h 
-btS^, %<7)AyP7li, r e c ogScOJKttt L 

[0 12512.4. 1 onReco:;^<>l 
f4, B«Ba66 I 7'9^1f(cttfflTS 6fgf^£*£#fci: 

# t/ZMth^tlt . automat ict- p' T"<7) r e c 

U UV— X£JH*-T4 (2.3. 2JfI) <, onR 
eco{4. ftSWfctt, SHHSS^Tu^AteX^ 

[ o 1 2 6 1 mx : 

[0127] 

C*l] 





■ Rcco onReco = "handler" > 




Object.onReco = handler, 




Object. onReco = 




GelReR"handler"); 



[oi28] jiyhxntxrhm 

[0129] 



C*2] 





feu 











[0 13 0] ^yf7Wf< : -f\yhAyp?|4 Bg) o 

TUJ^A t:fflg&\W.t>%:\*&. ^y^yitf—^iZ [0131] M: &.TCDXHTML7 o 

( -f <y h t/yi ? h Wfflffitoi J4I3lTcOM£# 7° 1 £ DftXHJ L , MW%y J H fc«£W 0 ST «, . 

< input name="txtBoxOrigin" type="text" /> 
<input name="txtBoxDest" type="text" /> 
<reco onReco="processCityRecognition() "/> 
<grammar src=" /grammars/cities. xml" /> 
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</reco> 

<script><![CDATA[ 

function processCityRecognition () { 

sml Result = event. srcElement. recoResult; 
origNode = 

sml Resul t . sel ectSi ngl eNode ("//or i gi n_c i ty") ; 

if (origNode != null) txtBoxOri gin. value 

origNode. text ; 

destNode = 

sml Resul t . sel ectSi ngl eNode (V/dest_c i ty") ; 

if (destNode != null) txtBoxDest. value = 

destNode . text ; 



]]></script> 

[0132] 2. 4. 2 onSilence:onS 
i lencefi, Reco^Oi ni t i alTimeo 

u tBtt^§ii^Si«n (2.2.1 £$fg) <r> 



aticrecognitio n^E— K^W^CfJBKT 

[0133]ii: 

[0134] 

C*3] 





<reco onSilenre- handler 1 ' ...> 




Object.onSilence : handler 


(BCMASeript 


Object.onSilence = 




GetReffhandler"); 



[0 13 5] ^yWy'i^hffii: 
[0 136] 



[fft4] 













status = -11 



[0137] j\>f7n^f< : A ^yh^y^yii 

[0138] 2. 4. 3 onTimeout 
onTimeoutll *IWC(it^75 7f7t- 



• iXj&J&^T?" h WlZMl t^otmaxt ime Rtt 



■ Blt^ttS^fc i 6 , b a b b 1 e T i 

meout (2. 2. 1 £#Bg) £ J; oT^gSiXJ^B* 

( i i ) . 

[0139] u^WMl fgfiMa^gww^JRD 

aw. 

[0140] 
[*5] 



-f >-j7^f yum 


■ rcco onTimeoulF'luMdler" ...> 




Object oflTimeout = handler 


(ECMAScript CD^r^) 


Object. onTimeout. = 




GetRefThandler"); 



[0 14 1] -fOKtTVx^Mf*: 
[0142] 



[*I6] 
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ft L- 




















*t 7 * ji> \- tfrft 


reco status ^-12 ^s^>£ 



[0143] -l^yhTnAr-f : A ^y h/Ny F 5ti 
To-KfM *itSfSttIX653r^36*. -Ay F^Jir'-^ 

[0144] 2. 4. 4 onNoReco:onNo 

Re c oXiMPW&rv -r yy*-Mzk->x¥k^ t>ti 



statusn-KCiotrailMI., WO 

[0145]lt: 

[0146] 

[*7] 



-I* >"7^f ^ HTML 


<Reco onNoReco^handler" > 




Object. onNoRjeco = handler; 




Objert.onNoReco = 




GelRefThandler"); 



[0147] A^yhyt?^*? hmm-. 

[0 148] 



C*8] 

















status -fust-T-j 4*tis?U null tg^J^irTo status 








status -13 : »W^ffl^^fr.^#^|He-c^4v\ 




status -14 : ffibi>*D&?ti>lfam£tlffl&. ZtlfcAK * 








2 2 1 <Z) reject JRf£&$i&) e 




status -15 : ^^m^^m^^/^^ ff J&«z>*fetii 




£ babbleTimeout JRffc^S^nfctW^rai^W^ 




^4S**rM+-t^#*^ofc (2. 2 1 



[0149] -f ^yFTn^vM : bA>-P5(l 
[0150] 3 prompt 

[0151] promp tg*{i. ^yi/W? 1 ) 

I Lrivy HfcioTSift-fbS^S ) o 
[0152] 3. 1 promptOTyrVy 
p r om P t]g*{3, f^xiiy-diH^r-f/l/^f 



^Mo'JV-xfc-fttr. #*Krp r om P ttt, OT<7) 

<prompt id="Welcome"> 

ACME ^StHRfcfc WSV ^« 1*1^3 r §"V * * 
</prompt> 

;«W/f^^ity OTt'ie»-rs««^v^n^ 

? r >y r * § iz^tbh i t ^T'# s . 
[0 153] 3. 1. 1 ?Tv7 

r o m P t S«^gET"ffifflt-§ i 3&*T* 4 . ( 
t-v 7 Ml 3. 2. IteEttL-fc rt t sj JHtT 

£ ; 1 36*tr# £ . ) JiTRowti . wait:* s — se 

<prompt id="gi veBa lance" > 

□.ffitC <emph> 5 K/k </emph> M^feDit. 

</prompt> 
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[0 154] 3. 1 . 2 Wm^VfW 
promptfliioayf ^ytl prompt^ 

a i u e mm^am-t^ z t #t$ s . 

[0 1 55] Va 1 ue^lR 

value: ffiKaHK. K hMZfo&Wmnm 
[ 0 1 5 6 ] ae : 

■targetElement: ff*mfK. hreft 
fcti targetElement <59V vf ;Jx:fr.£f^-f 4 
it t ?X'9til§ttSf(^-^ti'^*<7) i d. 

• targetAttribute : HMMiR. ffl^'ffX 

■href: ffiUK. Wl&-tr;JO< y FOUR I „ h r 
efttargetEl ement (T)Wft1f$ft£t %>t% 
■q- s href ^targetElement t-ffiTfe^tt 



[0157] targetElement JRttti. #tf 
d^'t argetE 1 erne n t l,Z X -^Tfe/ESftT^ 

(prompt) 

(value href="/wav/beep.wav" /> 



targetAttributelMfifflL 
T . targetElement tofcgSrJRtt *ffiS&$- 

iifcj&frrsi. ^t^.KHTMLfj»f» 

P r omp t* s iti7j$ix4ltir^ TtxtBoxOri 
g i n j J:tX TtxtBoxDestj SS?<7> 

r v a 1 u e j JR'ffi^f-^f X MCff XZtlh . 
(prompt id "Confirm") 
fr<5fe« 

(value targetElement "txtBoxOr igin" 
targetAttribute "value" /> 

(value targetElemeut "txtBoxDest" 
targetAttribute '"value" /) 

(/prompt) 

[0 1 58] 3. 1 . 3 #^7r^t 
value S*{±, -&&$tVfc prompt COiXfo 9 

fo&^tzco^Tn^.-t&titbiz. msizn^^ti 

£JlT<0#lTf i , prompt^DCt"-7t^5t 



(/prompt) 

[0 15 9] 3. 1. 4 #B98^7°ny7°r- 

^>7->v*-<{ >=74 yx-i'&fet&fth^iz. ^cr>mm 

t«ifcsrcItt*ttit4:i:T. OT^i 3 (c, 
U R I LT!?h§|53 yr^y i t #T# 
4. 

<prompt id="Welcome" src="/ACMEWeatherPrompts#Welc 
ome"/> 

s r cJRftCD^— y>y hit. -fy54>7DV/hHt 

[0160] 3 . 2 Ittiii^rn^f^ 

p r omp t®S<is OT^JW* (^yU-^y^^ 

-f') tsitfTn^-r-f fj'^yt/^tfe 
/t77^f) 

[ o 1 6 1 ] 3 . 2 . i ae 

■ t t s : ffiKHJR. htPt> ! &?l'^>&&£.m? 
&~?-7T<yTnm?4 7\ 9?*)Vh<r>m± r SAP 
I 5 j . 

• s r c :^y7-( y7ny7h«£§ri^(ffi 
SSI. «^7n>7t«URI (3. 1. 4 
SI) . 

■ b a r g e i n : ffiRHR. £&. 7ny7 r<7)|M4& 



tf)i*flSH* . f7 * ;P h ffymtW&X-h 9 . b a 

r g e i n (iffifflTS Bargein = 0 

(S, fiPB#barge i diNbTM. ^Wl 75-y 

h 7^-^36*ifJif- h ltv>4 t'^aaofijDa^t 

K^JtiiX*/^— <— XCOb a r g e i n^C0t*t> 
^ *«ffiffl^"|gTS> 4 J&HCJC LT„ V vftUHfi - OTTffiT 

•prefetch: ffiES#L ^—'Jftu—VZtltz 
fc j* fc , prompts >7ifflTfiB^fc^*5 «£ V 
-r -y >-,x-£-4'x§T&4* 1 £':) ^Sr^rrf , 7"— !V7 7 
7\ f7^)Vh COmt f a 1 se?M. 

[0 162] 3. 2. 2 7°D.A°-f -f 
T-yTP^y^^ifii. p r omp t<5DD OM*7y 
x ? h fltOTc07PAr -f §r^r^- h^S . 

•bookmark: SEKO^fffl. &±Lfc*3rO£j£ 
7" cr>^X h 45^9!* 7">> x ? 

ho 

■ status : £H5t OSffl. ^7°5 -y t 7t-At 
J:-5TMStt4 status a— h\ 

■ Innertext : StKD^ffl. Z<7)7rv*7- 4 
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ti. prompt CDt^-X bil^IflM ( t r a n s c r 
i P t i o n ) fcSfcU Ztvtf&mSMtmbtlh. 

fcbtlf. prompt tfniWfcy r 4 )V<rm%& "ttf 

l Ic7)Ta^N°T' -f H-tCOp r o mp t CDir^f X WS' 
fcffiftU acV^-CfctABT. prompts 

x. ttztitfr. j to~c%im.imm$fi&« i nn 

e r t ext/nrtf^lt BW^y-r yy£-^tf p r 
omp t«f^ h^-yg y^SfttSiCfcffiffit 

[0 16 3] 3. 3 prompt^V-yK 
promptB4ll promptWDOMt/y'i^ 
b C2fc h \8T<n* V ■/ b £ ffiffl LTMflTTS £ fc tfTS 
i^^TjS-Cfi, T-yTL-^^r^^-r*^ prom 
P t 3.9 Y<nm& (start) J; Wit ( s 

top), jiff 4 J <50 prompt COS^fWik ( P a u s 
e ) teXtmm (resume) % ts: hlf^^ilfz 
t^OMiJit/flWM (change) £Hff$- 
& 

[0164] 3. 3. 1 Start 
prompt f)B±£|£S&-$- & . B\WLtfi^-i hiV%\ W 

^.—£7) p r o m p t ^-y S^x ^ b ^&#}5r5-<£>B£IIIfc: 

r^teS^j Start^'iliL 
[0165]it: 

Object. Start ( [strText] ) ; 
• strText : ^J^ICMfiSft.Sx^r.X b <> ff 

tm-: s t a t u s = - 1 tCfSJg-rS k . W^N'.y ^ r 
Afc J; oT^fclfltk$*rO*4*£M:o n C o 
m P 1 e t e-4<yhjm!lhZtL&. 
[0166] 3. 3.2 Pause 

7 7r^77 -y i/V/tftStt- B^itf 

y >y b'(iiE^T S>S. 
[0167]«X: 
Object. Pause () ; 

<html> 

<title> 7°n>-7°bflJW </title> 
<head> 
<script> 



[0168] 3. 3. 3 Resume 

[ 0 1 6 9 ] «£: : 
Object. Resume () ; 
MOffl : 

[0170] 3. 3. 4 Stop 

-£\ ^ (7)^ y >y b ~!77Tj777yym £f 

[ 0 1 7 1 ] flfcfc : 
Object. Stop () ; 

worn-, 
mn--. 

[0 1 72] 3. 3. 5 Change 
B£<%att*i±tf/4fctt«£3GW4 . Chang 

[ 0 1 7 3 ] ffiX : 

Object. Change (speed, volum 
e) ; 

>U*~9 : 

- speed: f^M. ^$tt&\3Wi<, s p e e d = 2 . 
0 JiJjMECDMJSS: 2 imz-f h , s P eed = 0. 5im 
s P eed = 0IJf7t/lbf 
*Wt-$h kV^o MMXfo h . 

■volume: 3t35"$~& HI£ 4 v o 1 u m e = 

2 . 0 iiigftSO^MS- 2imz-t& , v o 1 u m e = 0 . 
5 iiil^^M^^tC-r^ , v o 1 u m e = 0 lif'7 
*)V hW&95ct& k^o MMX*$>& . 
MOffi : 
*U 

3rL 

[0174] 3. 3. 6 prompt c^MfP^J 
ETROWIi . ±IBC0^< y >y b SrffiJu LT , bfj 
S^Il*t^-MT^^^777h7t-ACML 
t, prompt coSflHI* t'Oj; d V V V7'X% 

hfr*^i-i>cr>x*fo& » 
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<! — 



function checkKWBargein() { 

news. change (1.0, 0.5); // «fS*#U a— i*£Ttf 4 • 
if (keyword, text == "") { // teR(iBWl*iST'*> S , 

news. change (1.0, 2.0); // sj? U jl— A^TUtM^o 

keyword. Start (); // f2»£BHW"So 
} else { 

news.StopO; // K^aiStlfc. TnyTb^M 



// 

</script> 

<script for=" window" event="onload"> 
<! — 

news . Start ( ) ; keyword . Start ( ) ; 

// 

</script> 
</head> 
<body> 

<prompt id="news" bargein="0"> 

[ o i 7 5 ] trbih. mm&mmm^immiz. a* 

tzti^tztzth, mi^tzt£^7*—^>xco$mzX^ 

- <! — 

</prompt> 
<reco id=" keyword" 
reject="70" 

onReco="checkKWBargei n () " > 

<gramraar src=http://denali/news barge in grammar. xml /> 
</reco> 
</body> 
</html> 

[0176] 3. 4 Prornp t-f^yf 
prompt DOM^yx? MiHW^fJ 
Ifrtf— bLTfcO. f^Ay^li prompts* 

cows t ucfflj&rs - 1 tfTS £ . 

[0 177] 3. 4. 



51«V NTfl^T 2 1 5 6. 2 6t?ITO, 
7. 05*MyfTILT1 0866. 4 6t^ot 0 



[0178J1X: 
[0179] 



1 o n B o o k in a r k 


[«9] 




<prorapt onBoolanark- handler". . > 




Object. onBookmark = handler 




Object. onBookmark = 




GelRefChandlo-"); 



[0 180] -f^yb^yx^bffil 
[0181] 



[aio] 
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[0182] 4 Of7oAT^ : A^VYiWY^M 

[0183] 3. 4. 2 onBargein: JL— if 
<7)b a r ge- i n^^hj&WlSiflfcfcHB&Sifl 
4. (barge-in^f^yh^M^ 



V\ ) £0^<>"b./N>"K7£fi5£LT i k, barge 

[0184] 
[HI 1 ] 



-r^t7-f viiiml 


■"prompt onBargein- "handler "...> 




ObjectonBargein = handler 




Object. onBargein = 




Ge1ReR"handler"); 



[0 185] -fOHtTVi^Mftl: 
[0 186] 



[*1 2] 





?£ L 




barge in h 







[0 187] -f^yh-Ttm^ : -f-^y f^fi 

[0188] 3. 4. 3 onComplete:pr 



[ 0 1 8 9 ] flbt : 
[0 190] 
[HI 3] 





<prompt onComplete- 'handler' '...> 




Object. oriCotnplctc = handler 




Object. onCorriplete = 




GetReffhandler"); 



[0191] ^yWy'i^hffii: 
[0 192] 



cm 4] 





ft U 




prompt # 4. 2* ttS T "f" -5 




Sttftilfifift? l-^T ^ f> !£ status = 0 ^ St 5 

as, ^ 5 -ettrtitflf status £ _Ls£T??g je £ nr- j; 5 £ 



[0193] A ^y ■ A^yb^ybyii 

[0 194] 3. 4. 4 y7?7-?fciW<yb 

<script><! [CDATA[ 
var mark; 

function interrupt ( ) { 

mark = event. srcElement. bookmark 



mtht^iKtmVhh. onBargeinAyb'5 
£\ ^n— r m a r k j g£i££r p r o m p t F*J*T3f 
» LfcfWr coy izK^rT & X ?V7°h 

BBU ZC0 r m arkj COWPjE I WlSrfS^S fcft 
C r e c o C^MSIWft ( r h e a r d j ) rffifflSfl 



function ProcessCityConf irm() { 

confirm. stop () ; // ^^fVt^ , 7 7rl:77 7y^tSc 
if (mark == "mark_origin_city") 
txtBoxOr i gi n . val ue = 
event. srcElement. text ; 

else 
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txtBoxDest. value = 
event. srcElement. text ; 
} 

)]></script> 
<body> 

<input name="txtBoxOrigin" value=" S^T 
type="text"/> 

<input name=" txtBoxDest" type="text" /> 

<prorapt id=" confirm" onBargein=" interrupt () " 
bargein="0"> 

From <bookmark raark="mark_origin_city" /> 

<value targetElement="orgin" 
targetAttribute="value" />, 

<bookmark mark="mark_dest_city" /> t v if <5fcT*£>& Z t S*fMn|jL 

</prompt> 

<reco onReco="ProcessCityConf irm() " > 

<grammar src="/grm/1033/cities. xml" /> 

</reco> 

</body> 

[0195] 4 DTMF 

DTMFJBH-^x^b*fff6t6. Z(0*"7ifj:9 
u 7f 4 y?xA yx 9 vMtth z t &x*% dt 

MFt4}g|Wfc;S*lfci:Sfc. prompt^in 
Cbarge-i n A Ky h £ BHftS-fr4 Z 1 3&*C# 

WJ&:T9^1P2 1 6fc^f>7t-A'2 14t« 
[ 0 1 9 6 ] 4 . 1 ayj-yy 

■ dtmf gr a mm a r : A y^A VXSffltS)^, 

mi : ^f^btvvtym 

<input type="text" name="city"/> 

<DTMF id="city_choice" timeout="2000" nuraDigits="l"> 
<dtmfgrammar> 

<key value="l"> y7h/l/ </key> 

<key value="2"> ^Xby </key> 
</dtmfgrammar> 

<bind targetElement="city" targetAttribute=" value" 

/> 

</DTMF> 

r c i t y_c h o i c e j j^ffiKrfKSftfc: i: § C . ^_ 
— i^s&s 1 2r#~t t r Seattleji\ 2 Srff"T t 
TBostonj ^A^^-Y-^KtCM^ST^^. ^ 

[0 198] 



■bind: DTMFSHIWa^aWSr^-f-^KtM 
[0197]Itt: 

■targetElement: ift£R 0 S^W&fSwfclfi 
ftWWOST^fLiB* (Zif:iJfW3C SMI L 

2. o H«) . 

■targetAttribute: IgMBWWO S 

2. 0<a*&fcH*> o ?7*)VY<nm$ r va 1 u 

e j 0 

■test: ffl3Ttf)*fr. *r7*)VY<?M\tt ru 

e , 
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#12 : »7>f-*PO»^«)DTMFfl)«ffl*tS 3S*LfcfctfOT*>4. 
{input type="text" name ="area_code" /> [0199] 
{input type="text" name "phonejiumber" /> 
<DTMF id "areacode" numDigi ts="3" 
onReco="ex tens ion. Ac t ivateO "> 

(bind targe tElement="area_code" f) 
</DTMF> 

<DTMF id "extension" numDigits="7"> 



(bind targe tElement="phone_ntunber" /) 
</DTMF> 

< input type="text" name="credit_card_number" /> 
<prorapt onBookmark="dtmf. Start () ; speech. Start () " 
bargein="0"> 
<bookmark name=" starting" /> 

</prompt> 

<DTMF id="dtmf" esca P e="#" length="16" 
i nterd igitTi raeout=" 2000" 

onkeypress=" speech. Stop()"> 

<bind targe tElement="credit_card_number" /> 

</DTMF> 

<reco id="speech" > 

<grammar src="/grm/1033/digits. xml" /> 
<bind targetElement="credit_card_number" /> 

</reco> 



fgSRo DTMF^UR 



[02 00] 4. 2 
4. 2. 1 «tt 

■ dtmf grammar 

I o 

[02 0 1 ] 4. 2. 2 Tdat^ 
■ DTMF g r a mm a r : fM5^/#2^ 
DTMF^itXML DOM Node^y'xn 

KM) o ?7*}l-h^m$£XFcOb&V)o 
(dtmfgrammar) 

(key value="0">0(Aey> 
(key value="l")l(Aey> 

(key value="9">9(Aey> 
(key value="*")*(Aey> 
(key value="#")#(Aey> 
</dtmf grammar > 
•flush 

saw bo m team t%&mB4yf~7*-A#~v± 

tDTMFA' 7 7r^g Md m tZ7 7vis>?'£ltZ> i: 



e 0 

■ escape 

iKD/ta^o DTMFiffiH7yHyB?7t§ 
tlbbCD escape — 0 escape 1± 1 

■ numDigits 

lf3RD/«ii^o DTMFlBROHry^HyfeiRT^ 
tW-XfO-^t, escapehlengt 

■ dtmf Re su 1 t 

A*Silfc«^lilSa«te s capemfL 
■text 

■ initialTimeout 
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■ interdi gi tTimeout 

[0 2 0 2] 4. 3 t7'yx^h^V7K: 
4.3.1 Start 

DTMFffBZ^tllffnriEtt. DTMFUHyy 

[02 03]«: 

Object. Start () ; 
MOffi : 

»: 

[0204] 4. 3. 2 Stop 
DTMFiffiffl^ffiKtSo fc*fU ^-iWA^Ufc 

[0205J1X: 
Object. Stop () ; 



[02 11] ^^Wyx^btSi: 
[02 12] 

.^yp 

[0213] ^yh7Wf^ : >f^yb7\yP5li 

[0214] 4. 4. 2 onReco 
D TM F ^ 7 y 3 V «7t S t »3 fl^ , ££)>f^ 



[0 2 17] ^f^F^yx^WI: 
[0218] 



[0219] <i^> h7nA'f : -f^y W^b^ii; 



£L 

: 

[0 2 0 6 ] 4. 3. 3 Flush 

DTMFA.y7r*757^tS. FlushliD 
TMF^ 7 y 3 V * £ fff Vffl^i 1 **CS StM. 
[0207J1X: 
Object. Flush () ; 

[02 08] 4. 4 ^f^yb 
4. 4. 1 onkeypress 
DTMF^-Srfff fcSR&SftSo ^iX«HTML$IJW 

if ^ escape ^ — SrPP<\ f: . on Keypress 
Tl4=Sr< o nRe c-{<yb«§^ fl 

[0209]«J: 

[0210] 

[gl 5] 



[ 0 2 1 5 ] fltSl : 
[0216] 

mm 



[0220] 4. 4. 3 onTimeout 



<l >1W onkeypress- todler" ...> 
Object.onkeypress = handler 
Object, onkeypress = 

GclRcfC'handler"); 

[*16] 



<DIMFonReco='liandler" ...> 



Object. onReco = handler 
Object. onReco - 
GelRcfl"handler"); 



[H18] 



^—f 4 ^ escape ^f— h 
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[0222] 


[ 0 2 2 1 ] mX : 




[S19] 






<l > IMF onTimeout-'TiaadLer" . . > 






Object. onTImeout = handler 




(ECMAScript £>#£0 


ObjecA.onTimeout = 






GelRef("handler"); 



[0223] -(^.yb^y-j^^hmm: 

[0224] 



[82 0] 



[0 2 2 5] •f^yfTnA'f ^ : 4^ybAyh'5l± 
[0226] 5 Callcontrol ^7Vx ? h 

tsxx/mm zm-t. ^w^^bd gui^ 

^1PT<7)w i n d o wt^y'i ? h b H t ± 3 izM^WT 

ft § . l*^t , iRtr^x 7 b «#^«§ra(±7- 7 
77>>ffi nf r t *c i of^iist/yi 7 b * y 

[0 2 2 7] £<9:fr:7Vx? hfc^rLTHSl 

[0228] 5. 1 TOAr 4 
• address 

MBtOWfflo XML DOMy~Wy'x7b, 5gJ& 
JBttfcioTa**. Wn^TYVXThh, PST 
NCO^ii. ANItALI COffl-^TTft o T± </■><> V 
o I P cr>Wr£, i tUW&t&O IPTHWtfti. ■ 
ringsBef oreAnswer ^ffB? t JE^"f"-l> 

n swe r ( ) ^ V -y F Srffiffl LSrltfttS'Sr^Sr^J: i: 

* & . mts^^'*fiof izxtiz. cDtiZA 

[ 0 2 2 9 ] 5 . 2 .X V >y K 

[0230] 5. 2. 1 Transfer 

y-x £**rt-£ - 1 ^"c# 4 . 

**: 

telephone. Transfer (strTex 
t ) ; 

^n°7^— ^ : 



■ s t r T e x t : j&l. mi&C0&m%<0T YVX. 
WOO. : 

f£, S**3¥#^1ST-^f«^\ i o L!tf^7 r «y 7 

[02 3 1 ] 5. 2. 2 Br i dge 

ffi3#*ES£. Df#IBai3*ufc«L 7"7'7-f'aPftcfiJ'9iS 

t rU I DS:ffifflLTMSix^i:Stc, t^yygy 

IX 0 WsW* **TS-£fc t § {zo^W?^ t £ . 
[ 0 2 3 2 ] fltfc : 

telephone. Bridge (strText, 
strUID, [imaxTime]) ; 

- strText : <&m„ mM^fS^T F 1/X„ 

■ strUID: 3E£CDHf£- Sfc»3W-ft-fe "V 
ygyi D. ISt$^Pf*fR$^-&i:t:C srtU 
I DAia d d r e s sS'ffit^$tLS. 

• imaxTime : ffiKHR. SBMS^D?C0#*ffi 
W«±»^B*r B 1. *KeS*L*V\»^, f'7t;l-hOlt± 
7° 5 -y h 7 *~~LP\mmX'foh . 
MOffi : 

[0233] 5. 2. 3 Answer 
telephone. Answer () 

worn. : 

o nAn s we r <>- btilHiSS^V^ 
[0234] 5. 2. 4 Hangup 
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act: 

telephone. Hangup () 

mvm ■. 

[0235] 5. 2. 5 Connect 

■■ 

telephone. Connect (strTex 
t, [iTimeout]) ; 

• strText : MM, fftMO'g.it^TYVX, 



• iTimeout 

worn : 

KWfa&qat&fl* (ft : ^-K?*TW:£<>D(Mfi£ir 
[0236] 5. 2. 6 Record 

a.— tr 7r^( /wcbmw-* . 

W 3z : 

telephone. Record (url, endS 
ilence, [maxTimeout], [init 
ialTimeout]); 

<HTML> 

<HEAD> 

<TITlE>Logon Page</TITLE> 
</HEAD> 
<SCRIPT> 
var focus; 

function RunSpeechO { 

if (logon. user. value == "") { 



■url: MM, m^tlfz^COu r 1 

• e n d S i 1 e n c e : jfiMH. mmtfffiiiiZtlfzmz 

•maxTimeout : fi JullfR 0 lBJS"f~ & tZ&bCD® 
■initialTimeout: ffiB3I#?o tfiSitOfe 

: ia»^'u r i tfia^t^ic. wm^kc. 

[02 37] 5. 3 >f^hAyh'5 
[0238] 5. 3. 1 onlncomingO 

^p7°y Wfimmwm-o tzt * en* t*ai 

[0239] 5. 3. 2 onAnswer () 
[0240] 5. 3. 3 onHangup () 
Hi. 7°0^"5A#'H a n g u p ±tziiT r a n s f e 

[ 0 2 4 1 ] 5 . 4 M 

iJ7f -f y^Sr^-ri>cOTS>4. 



dtmf. Start () : 



focus="user" ; 

p_ui d. Start () ; g_login.Start() ; 
return; 

} 

if (logon. pass. value == "") { 
focus="pin" ; 
P_pin. Start () ; g_login. Start () ; dtmf . Start () ; 

return ; 

} 

P_thank. Start () ; logon, submit () ; 
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function login_reco() { 

res = event. srcElement. recoResult; 
pNode = res.selectSingleNode("//uid") ; 
if (pNode != null) 

logon. user. value = pNode.xml; 
pNode = res. selectSingleNode( "//password" ) ; 
if (pNode != null) 

logon. pass. value = pNode.xml; 

i 

function dtmf_reco() { 

res = event. srcElement. dtmf Result; 
if (focus == "user") 

logon. user. value = res; 

else 

logon. pin. value = res; 

} 

</SCRIPT> 

<SCRIPT for="cal 1 Control" event="onIncoming"> 
<! — 

cal 1 Control . Answer () ; 

// 

</SCRIPT> 

<SCRIPT for="cal 1 Control" event="onOffhook"> 
<! — 

P_main. Start () ; g_login. Start () ; dtmf . Start () ; 
foe us=" user" ; 

// 

</SCRIPT> 

<SCRIPT f or =" window" event="onload"> 
<! — 

if (logon. user. value != "") { 
P_re try. Start () ; 
logon. user. value = ""; 
logon. pass. value = ""; 
checkFieldsO ; 

> 

// 

</SCRIPT> 
<B0DY> 
<reco id="g_login" 

onReco="login_reco() ; runSpeechO" 
timeout="5000" 

onTimeout="p_miss. Start () ; RunSpeechO > 
<grammar 

src=http : //kokanee 1 / etradedemo/speec honl y/ 1 og i n . xml /> 
</ reco > 
<dtmf id="dtmf" 
escape="#" 

onkeypress="g_login. Stop() ;" 
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onReco="dtmf_reco() ;RunSpeech() " 
i nt er d i g i t T i meou t= "5000 " 

onTimeout="dtmf. Flush () ; p_miss. Start () ;RunSpeech() " /> 



<prompt id="p_main"> f I Dt5 it^t^S^Sr fSoMLT < V\ </pro 
mpt> 

<prompt id="p_uid"> JL— if I Df3fWmLX < £f£ </prompt> 
<prompt id="p_pin"> b°y#^ff W11 n liLT < </prompt> 
<prompt id="p_miss"> $ Lf?S> 0 S iiX <> ^lALS Lfco </prompt> 
<prom P t id=" P _thank"> feW^^T^iL^ #S*flK^*HI*5f*fc< 
/if § </prorapt> 

<prom P t id=" P _retry"> #LR*^f^ JL— if I Dfc it^f^*^— & 
Li-BXo </prom P t> 

<H2>Login</H2> 
<form id="logon"> 

UID: <input name="user" type="text" 
onChange=" runSpeech () " /> 
PIN: <input name="pass" type=" password" 
onChange=" RunSpeech (> " /> 

</form> 
</B0DY> 
</HTML> 

[0242] 6 y^ruyy^-mm y^ruyyu-^mm^h^m^^th^hh. ^ 

6. 1 ^rn/7n-Jilf§f:WHTML^ ;l/77nyrb(7)ayfy7M^M(t^^^t:HTM 
i^x?'j7T«fl«l A^tf>?xt;:A;fU> LA^f«« ( r too l t i PJ mmt Lx^yy^ 

mzmL, x-fim^wm^vT^m-h, mm ^rx-mmtih) t\ titi eittiffi^ts, 

<htral> 

<title>Context Sensitive Help</title> 
<head> 

<script> var focus; 

function RunSpeech () { 

if (trade. stock. value == "") { 
f ocus="trade. stock" ; 
P_stock. Start () ; 
return ; 

} 

if (trade. op. value == "") { 
focus="trade. op" ; 
P_op. Start () ; 
return ; 

> 

//. . y 4-)VY^X{z-jY^TAL^mTM1r . 

trade. submit () ; 

} 

function handle () { 

res = event. srcElement. recoResult ; 
if (res. text == "help") { 

text = "fI^LT<^$W' ; 

text += document. all [focus] .title; 

P_hel p. Start (text) ; 
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} else { 

// mwmx zwn^h . 
} 

} 

</script> 
</head> 
<body> 

<prorapt id="p_help" onComplete="checkFi leds() " /> 

<prorapt id="p_stock" onComplete="g_stock. Start ()"> ftx^^^rfS^BbT 

<^i$V^ 0 </prompt> 

<prorapt id="p_op" onComplete="g_op. Start () "> 9 "C"^** . HW^fr 

0 </prorapt> 

<prompt id="p_quantity" onComplete="g_quantity. Start () "> MttT^^ 
0 </prompt> 

<prorapt id="p_price" onComp lete="g_pr ice. Start ()"> fiHffS(i:V^< £>T"f" 
fpv </prorapt> 

<reco id="g_stock" onReco="handle() ; checkFieldsO " > 

<graramar src=" . /g_stock. xml /> 
</ reco > 

<reco id="g_op" onReco="handle() ; checkFieldsO" /> 

<grammar src=" . /g_op.xml" /> 
</ reco > 

<reco id="g_quantity" onReco="handle() ; checkFieldsO" /> 
<graramar src=" . /g_quant. xml " /> 

</ reco > 

<reco id="g_price" onReco="handle() ; checkFieldsO" /> 
<graramar src=" . /g_quant. xml /> 

</reco> 

<form id="trade"> 

<input name="stock" title="stock name" /> 
<select name="op" title="buy or sell"> 

<option value="buy" /> 

<option value="sell" /> 

</select> 

<input name=" quantity" title=" number of shares" /> 

<input name="price" title="price" /> 
</ f orm> 
</body> 
</html> 

6.2 smi Lcomm 

jaT^OWli. SM I LHMU:p romptSitfre c oB*oiSKr 

<htm 1 xml ns : t=" urn : schemas-mi c rosof t-com : t i me " 

xml ns : sp="urn : schemas-mi crosof t-com: speech" > 

<head> 
<style> 

.time { behavior: url (#default#time2) ; } 
</style> 
</head> 
<body> 
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<input name="txtBoxOrigin" type="text"/> 
<input name="txtBoxDest" type="text" /> 
<sp:prompt class="time" t:begin="0"> 

</ sp : prorapt> 

<t:par t:begin="time.end" t : repeatCount=" i ndef initely" 
<sp:reco class="time" > 
<grammar src="./city.xml" /> 
<bi nd targe tE 1 ement=" txtBoxOr i gin" 

value="//origin_city" /> 
<bi rid target.E 1 ement=" txtBoxDest" 
test="/sml/dest_city (@confidence $gt$ 40)" 
value="//dest_city" /> 
</sp:reco> 
</t:par> 
</body> 
</html> 

[0 24 3] 7 SMEX(Message)gt/^ 

Simple Messaging EXchange 
/EXtensionWfiWt'fcSSMEXIi ?7 

mmm &tii±ry°v-!r-i/3>t mm? h 

TfoZ,, ^/*< s me x>i«itB*fc LT. XM 
L 4 (i|WI«C0 V— ? T -v T^—X r-'^f^yb (cS#> 
i&tf-r i:* 4 T'S ft . ^ ■y-fe-y'y^^yi ? h^M^ 

tWitt. its. 7 h H\ V- ? T -y 

[0 244] >f yx^yx-fb-TSt, ^Sot^M*, 
* cofll j£> * 7 ^ - ^ 4 f i«tt«f * it L X , T 7 'y 

;xtm lvalue),^, 

-yfe— >-'£ffifH-&. XML DOM N 
o d e ^'f 7WOAr -f t) ffl, , messaged" 

fill : smexJn^^7yx?htUfflffl« 
<smex id="logServer"> 

<param name="d: server" xmlns:d="urn: Mi crosoft.com/COM"> 
<d : protocol >DC0M</d : protocol > 

<d: clsid>2093093029302029320942098432098</d :clsid> 



So 

[0245] message 47"cfi s m e x^/yi^ 

MX'te. s me xt7y'i7hiiIf£vXMLi«i7 

smext/y'x^MiWXhF^fa^y 
h#Tyn-K$*t£i:fgSt£:fl&<, ^< sm 
e x^'vx y- Migift? 'J-yT^T^trt, T>- 
O- H § b $ izmm V V -X £ JBSrf & i t 

[0246] sraext7y*x^ Hi, y< i^'<7)® 

{4, ( S I Pi^fiCCXMLTffifflSftT^&J: 3 
W«Ii^ 7t-y7=f-7-yr-^<l»n§ 

mm^MLxwmi-^^b^mt u^*&**fts. * 
sttT-^T-y^-Yfi. b7t- ^trrn-y 

[0247] 
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<d: i id>0903859304903498530985309094803</d : i id> 
</param> 
</sraex> 
<listen . . . > 

...// Ai7^/^'Orec ofeft*tt*^4ffi<9ffi* 
<bind targetElement="logServer" targetAttribute="sent" 
value="*[@log $ge$ 3] "/> 

</listen> 

[0 248] zcomii, dfciWy^-7x 

y & gcD ± o taaw- h n* l got* 
cowcfi. rry^yHyiMfi, m—<7)b i nd 

ffijRJt (directive) ^ffifflf S :i£J:o 

[0249] dtfOMli. sme xt/^x^ h s^lSSS K 

<input type="text" id="remote"/> 

< input type="text" id=" transfer "/> 

<input type="text" id="local"/> 

<input type="hidden" id="session_id"/> 

<smex id="telephone" sent="start_l istening"> 

<param name=" server ">http : // tel -svr/whatever</param> 

<bind targetElement="session_id" value="//sid"/> 

<bind targetElement=" remote" value="//remote_addr"/> 

<bind targetElement=" transfer" 
val ue="//transf er_addr Y> 

<bind targetElement=" local" value="//local_addr"/> 

</smex> 

[0251] z&mn, 5JtiRofc^ y^^immt 

Ztzisblzb^Xolzb i ndJg^5t^ffiffl"T436^^L 

i.XfU—ij)VTYVX^mt, f7'IIremote_ 

addr, transfer addr, loc 

a l_addr*ttSfcfi£LT^S 8 
[0 2 5 2] dcOMT(i. •fSif—^^fca^-Sfcft 
t . HTTP Clo' < ffi^^o ii^nbn /l^WSffl $ ft 

fc«m^4<k3CKff;3ft*fcft. &7y4T>Yl±, 

0 S>ft3teHWtf> i d7mBtt£»ML£WUP5r 
£>%rV^ ^Wt ^—JMz r s t a r t_ 

1 isteningj^ v^^i/^i%iE'$~Z> — b tC j; o 



*KW LTMSW~S It^i«l^t777f7^ 
±iE£>Mi±. »J«SHft&*. *ft-Pft**ttStf>sme 

^ - y x - x ^ mirr s <r t «r#gT*#> s d t & hi iz^ 

4 T C P *r-J $ ft S • 

[02 50] 



£0 tffifflT'^ S Web Uw*fc3giH LT . 77° U ^ 

d«L7 4— ;l^fc*8ttSft£ 0 recoO^iR 
*^^l)7°77f7^-A^7^y(:SUJ) 

OMIJ . SW^S^fc. BWcOi dtfSJtffifeft 
-6 fc'frFC* S <! t * Bf fc^ LT V * £ fc cTrriiSr V ^ . 
[0 2 5 3 ] 7. 1 Tn^TM 

■sent: fnjJR "9 /W&fcVfo r> T . 7°77h7t- 

j±m$Mmzmt$ti&j< y-fe— s^twj&f ss^i, 

sent^lvaluet UTffiffl 5*1.4*^(4*^. 
W^yfy^tfrUA^yf (dispatch) $ 
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tLt. Toy If < i* r v a 1 u e t Ltfiffl§fl4 * 

wtw*r <fcnui l ^y s J^ b&m 
*} %X ^a&^r«i«jJT* 4 . 

■ received: Il^fltft^t. ^itKofc 
DOM Nodef-^. ^ 
•y-fe— i^ii, S<OonRec e i v e -f hcOJUfpp 
ffi**T'# £ 4 X\ r v a 1 u e fc LTffiJOTgfc % & . 
■timer: MM *) /Wk&X'fo -r> X , AT 1 /* h 

tf>8c. 7nA"f -f KEtfjgbWW 0 ST £>fl& 4: , ? a >y 
? B*W* . iJ V > h >cr>Wft +fc, 

at^ h b * h y n ■>/ ? ^ffit^ * 



status : 



t/tr? -y b 7 or -A fc crmmi>m±X' $ ^ vd»* sfcli 
C0X°7— ^ >yfe— i^Jis receive d7°oy\°f- -f £ 
tf«i<i*»§;fx£:*£-, s t atus3-HliO?ft 
[02 54] 7. 2 

<smex id="logServer" ...> ... 



-onReceive: CltD-f O'Mi, y 
-A^>y-fe-^WJ^LfcfctfcjMft£rt£o bind 

MSttS. iMfiCOHUfc, rece i ve d7n;ff -fi s 
- o nE r r o r : ,I<7M ^> H±, ^ -AT 1 /- f-^iH 

T L/c fc § , ttzimmv >7^7-lzM& Lfcfc £ fc 

mmzn&o zcveohtfmiEZfx&b. statu 

sews*.*. 
[ 0 2 5 5 ] 7 . 3 

KR^JB»SaSS&f-4«^, s m e xi±OTO^-S*S- 
i fc j6*T& 4 . 

•bind: Sttffiofc.* >y-fe— S^JiTJg^^t^fPL 
•CV*5*8£-£|&#, r e c o fclBpRL 

■ param: rec oOiftp^fc [cll^lifc, s m e x^"7' 

|i, l-pararaSitli r namejfMfifflL 
X^i-6 £ fc ^fg-CS) "3 , para mS«<?)rJ yf 

[02 56] 7 . 4 fflioa^ 
tr^fV^WjBfflfc sme x^llts^Mo l o 

</smex> 



[0 2 57] <lfL{4, 
IBS: (^O-A^) | 



<script> function logMesseage(logClass, message) i 

logServer.sent = logClass + "!" + message; 
} </script> 

HBSfcti, m*fe1ff]?^~?AX^ 
-thi> cox'fo h . ±f E^Ti 4 , n $f> ?"«4 , i d 
M-& i 3 fcTn?'5A§;fx& . 

<smex id=" logServer" onload="addFunction()"> ... 
<script> 

function my_logMessage( logClass, message) { 



[0 2 58] ^n— A;PBBR£#a2rVvA.<*>»£\ EC 

MA Scr i p t CO Tprototypej 7W?f -f 

^ffiffl tx . ^y-Jx. 9 hxv -y K t Lxmmzttim 



</smex> 



logServer.sent = logClass + 



+ message; 



function addFunctionO { 

logServer. prototype. logHessage 
my_logMessage; 



I 

</script> 

[0259] x*)*?^**.? hmmcrmirfT&vcomm. 

Ii» ISlTiO J; 3 tc»f i fc j&^S S . 

logServer. 1 ogMessage (REC0_L0G_KRR0R, "My 
message") ; 

[0260] ^<.xco£^mmte~txizmmx-miLZ 



S^SiiS ; fc fcWE^rtfcV^ 

[Hi] 3yta-f^ vtfft^ ^mwmm.<?>m 1 
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[02] Mi<oay\fjL~r- < y^f^-i xcom^MX- 

[04] iHffl3Vta-^<o«6KHrfc4. 

[05] 9J4T>\>/-V—rtisX7-J*l&(n7- ! *T9 

[06 ] yii-Yfflk*mttht&><?mmk 
[07 ] ryhmx-mm^c^-^r^mm 
[08] «stij:txffwawiifi**^-5. ?nry 

Xfoh „ 

[09] wm^wmiixr/^mt^xf-Mz^h, 
m^-tmx'hh. 

[010] ^imm^Kmv c ±mmti t ^x'rMzh 
& . v^-iryy mx-mr^m^ v- 9 r >y -rwmco^ 
- : JM*m-tmx*fo%> <, 

[0ii] ^mM^mmni.x/^mmwm.^, 9^ a 
ryh wfcmmm**?-? r >y rii<^-y«s ^ 

[012] ^ffiffi**fflfc «t t^awta^-^ ? 7 >r 
[013] -f-AfsiT^w y^i/jL-Mzx-oxmft 



[014] BKlfwtofS 1 coiM^E- K fcS*f*£BB 
[015] BKlJw S£7)H 2 <3iWFE- K * *-TMH0 
[016] BBlfw * ^ 3 <0»fFE - F £ ScT 3&H0 

[017] x^urh^t-c^^-fryhii-cwf^rsg 
r -y rmo^- ^MSr^-r arc* & . 
[018] xrvyh&L-cfjJ TyhMx-mft^lfe 

8 0 *«^r«IS 

202 Webt-A' 
2 04 

20 5 -fy?-*-yh 

208 PSTN 

210 $Z?Vo I P 

2 11 W^^t'CORe c o 

2 1 2 fljgifjsy^-f 

2 14 X -rV T^w* ( 'J h "llfc*:/^* ? h ) 

216 I Ef^77W 

307 TTS 

309 l«^C»W^B 

320 asp+^- v'W^iBMfif /-r yy°\s~h 

324 f'A myfflw 



[0i ] 




35c 
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[02] 



[03] 



RAM 



43 



30 



29 


37 






A/D 





59 



7 










D/A 






[04] 



TramT 



3^ 



152 



14 

185 



J4. 



3T 



160 



80 j 



-i-tfA7: 




190 



[ I 87 



IT 



192 



Hi 'J T 



v -f <5> □ ^ jt > 



■J^-H 



7*aiOA 
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[05] 



200 



324 



320 




202 



-„ / 



Webit 





^ -v 








ttMKB 

^ — J 










Web ! 
It- hf* 





205 









H 


I RAO 1 





30 



SAPl +tw? 



Reco 



Hit 



212 



^211 W ^" A 1 




309 



204 



[H6] 



250 



252 



T 



254 



"264 
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[07] 



260 



262 



<html> 

<rorm fd - "g et_ca rd_inTo" method = "post* action = H http:/ /pay m ent.asp'> 
<select rtame^"card_type"> 

<optfon va kie- "a m sx*> American Express </optlon> 
< option valu&-"tf isa"> Visa </optlo n> 
* <option value = "m s">MasterCard</option > 

</setect> 

< Input typs- H t«>ct" name^ B card_nuni" width=*3o H 

onCh ange- "handle () 11 /> 

< Input type= "text" name = ~oxplry_date" /> 

•sinput type= "subm li" value - "-Sub m it" onclicks "verify [)" 
<:/form > 
<scrlptxi[CDATA[ 

function handle O { 

it Cget_card_inf o .card_type , vaTue -- "amex") ( 
if (get_card_lnf <j . c«rd ntim , length" I - 15) 
alert ("amcx should have 15 digits"); 

] else 

if | get_card_inf o * card_num, length I = 16) 

alert ("visa and master should have 16 digits") ; 

> 

function verify ( ) { 

var flag «• window, confirm ( "submit ths credit card info?"), 
i£ (Tlag) 

gstcard^info . submit ( ) ; 

> 

]]> 
</script> 
</htm l> 



[01 0] 

function _}iandleO { 
handleO; 
checkFilledO; 

} 

function checkrilledO { 

if £card_ type. value = ,tw ) { 
p _card_type, active O ; 
g_card .types .activate O ; r eturn; 

} 

if Ccard_num. value = { 

p_card__num.activeO; g_card_nuirt.acti vateQ; return; 

if Cexpi ry__date. value — *") { 

p_expi ry__data. active Q ; g_expi ry_date.acti vateO J return; 

P_con tent, acti vateO ; 
p_conf i rm . activate Q ; 
confirmation .activated) ; 

} 

function conf i rmed(gobi) { 

if Cgobj . recog Res. text — "yes") 
^ get_carcLinfo. submitC) ; 

// user codes start here 
// 

function handle Q { 

if Cget_card_i nfo. card_type = "afiiex") f 

if Cget„card_info.card_num. length ! = l!>) { 

prompt , speak Q"aa\ex should "have 15 digits"); 
get .card .info. car d_n urn ~ 



</scnpt> 
</body> 



se 

if (g eCcarcLi nfo. car d_num. length != 16) { 

prompt .speak C'visa and master should have 16 digits"); 
get_card_info.card_nuin = 
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[08] 



270 



290- 



2/2 



<html> 

<form id= 1, get„card_info " method = "liost" actian = *htt|i://payinent.asp fl > 
<select name = "card_type" onCJick= "tal^O-Ca rd_ty pes)"> 
<optlon valus="smex > American Express</optlon> 
< option va]ue = "vfsa"> V isa< /options 
<optioii value- "ms^ MasterC^ rd < /opt ion > 
</select> 

<reco id = "g_ea rd_fcy pes" on Reco - n handle() H > 

- < grammar src = \/ g ra rn #ca i d _ty pes" /> 

</reco > 

<lnput type = "t^xf namc^*card_num' iArJiTth = "30" 
onCJick= "ta lh(y„C3rd„num )" /> 
<reco id=" g_crfrd..num " onReco ="handk(J v > 

< grammar src = " ./grain digits " /> 
</reco > 

< input type-"texf name- 'cxplry_d*te n ■ 

onClick^ n taJk{o_expirY_date)- /> 
<reco id= ' g_expiry_date " > 

< grammar src =" r /g r am#dates " f> 

</reco > 

< input type- "su bmrt* value-"Subra it" onClick^ "verify ()" /> 

</form > 

<script> < ![CDA f A[ 

function talk(gobj) ( 
gob j , ^cti vg t e ( ) ; 

) 

function handle ( ] { 

if (get card info. card num f= null) -I 
if fget_card_inf o. card_type . value 

i f (get_c^rd_in^o. carci__num . length 

alert ("amex should have 15 digits"); 
} else 

if (get_card_iiifo.eard_num* length != 16) 
alert ("visa and master should have 16 digits"); 



— 280 



— 282 



— 284 



t= 15) 



) 



} 



function, veiify () { 

var flag = window. confirm [ "submit the credit caid 
lofa?"] ; 

if (^lag) { 

ge c_c ard inf subfiiit [] ; 



} 



< /script? 
</htmf> 



(*13) 103-1 3 1 772 (P2003-131772A 



[09] 

< body > 

< form ld=" gat_ca r^^info " meLhud ="pp*t" action-" http://payment.5sp N 
onActivate = " welcome) "> 

< prompt id |> welcome "> W e rtovu need your credit ca rd </ prompts 

< prompt Id = " p_mum bla "> I didn't understand y«m </ prompt? 

300 < prom pt Id -"p _c»rd_tyft* " b arg e in = " tru e " > Wh*t credit card would you 

V un? </promp)t> 

< prompt id =" p_card_num " baraein = " true "> Plesse say the numfcer" </ prompt > 
c prompt id p_expiry .date ■ bargeJn = " true > W hat is the explrstld ii 

date? </ pram pt > 

^ » — '< prompt id =■ p_coi>tent "> 

303 I have your < value select -" csrd_typ«s " /> < velue select card.num "/> 

with expiration date 
<: value select =" explry^datc " /> 
<f prompt > 

< prompt id =" p_eoiiffrm "> Is this correct? </ prompt? 

< reco id =" £_card_types " onNoReco =±- „iunt ble(thl», 2) " > 
onfteco -" _ hand l< (this, c^rd„ty pa) " /> 

< grammar SrC - " ./ g ra m #card_ty pes " 

</reco > 

^ reco id=" g c«rd._..um " onNoReco - » m u m b le>(th i=, i ) 
onReco =" Jh3ndle(this, card_numj " J> 

< grammar sre ^" ./gram #digi:> * {> 

</reco > 

3Q5 < reco id = ' g ex p i r y _da te n onNoReco =" mumble(t!iis, 1) =» 

onReco = M _han™ l«(this, expiry_date) * audio riu " /> 

< grammar sre - * ./9 ra * rt *di a *«» " /> 

</reco > 

< reco id = " confinn ation " onReco »" conftrmed(thls) H /> 

< grammar src= H ,/gra i*i#y«sri€S " 

</reco » 

< select name -" eard_type "> 

< option value -"BmIBxV American Express </ options 

< option value - "visa "> Visa <{ option > 

-s option value M ms*> MasterCard -c/ option > 
</ select > 

< input type - " text" name - " card^num " width = *3o*/> 

< input type - " text" name = " «xpiry_ uate rt /> 
<: Input type = " swrbm it " v a lue - " S ubm it * / > 

/ fo rm =* 
< script ■> < ![CDAI A[ 

function welcome O ( 
p_welcome * active () ; 
repeat = 0; 
ehe chilled O ; 



302 



function numble { g ob j , nixprompti > ( 
gobj .deactivate {} ; 
p pmntolft . active ( ) ; 

checkFil led () ; 



i 



110^ 
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[01 1 ] 



<body> 



<form id= n get_cai'd__info" method = "post" actiori= , 'http://payment-asp'' 
onscti vate— n wefcome() n > 

<prompt id="p _welcome">We now need your credit caid</prompt> 
<prompt id ="p _mumiiie n >I didn't undcsrsraHd you</prompt> 
< prompt i d = " p_ca rd_ ty pe " b2rgein= - true M >What credit card would you 
use?</prompt> 

^prompt id= r >— canJ _ n,j m" bargein~"true">PCsase say tha mimber</prompt> 
< prompt id="p_expiry_date" bargein="i:rue">W!i^t is the expiration 
date?</prompt> 

< prom pt id = "p_co rrtent" > 
I have your <va!ue select='card_type" /xvalue select= n card_rtunT [> 
with expiration date <vafue se i e ct - "ex pi ry_d ale" /> 
</prompt> 

<prompt id= M p__conf[rm M >Is this correct? </prompt> 

<reco ld= M g__get_eardJnfo" onReco=s*_handle() H onNoReco ="mumbleCtiiis) h > 
<grammar src ^''./gram/getPayment 1 ' /> 
<bind target = "card__iype" valLre=Y^ard/type" /> 

irjc . <bind target ™ "ca i d^num" value=Vcard/numbci" /> 

* uo <bind target^ "expiry_d ate" value = Y^rd/expr^date" /> 

</reco> 

<reco id = "confirmation n onReco =' confirmed(thrs) onNoReco-"mumbI^{this)" /> 

<grammararc = /grarn #yesnoV> 

<frc\ca> 

< select name- eard_type > 

<option value=' , ame)f">Amei ican Express</opLion > 

<; option va lue= "visa " > Visa </option > 

< option value= "ms " > MasterCard </option > 
</sclect> 

<input type="texf l name = "card an urn" width="30" f> 
<;rnput type="text M n^me = *r±xpiry_date" /> 
<input type^submit" value= M Submit n /> 
</fbrm> 

<scripl>< J [CDATAT 

function welcome Q { 
P„wel come. active O ; 
repeat = 0; 
chackFilledO ; 



unction iriumLI g Cgouj) { 
gob} . deactivate O ; 
p_mumble .active O ; 
checkri 11 edQ ; 



: unction _handle() { 
handleO; 
checkFilltidO ; 
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function checkFilled() £ 
if Ccard__type. value — "") { 

p_card_Type,acLiveO ; ; do„fi eld. activate Q ; return; 

if Ccard^num. value =^ ,,M ) { 

^ p_card„num.acl'iveO; do_field.activateO ; return; 

if (expi fy__date, value { 

p_expi ry^data.activeO ; do_fi>ld. activate O ; return; 

p_content . acti vate O ; 
p^cenfi rm. acti vate O ; 
confi rmation . acti vate O ; 



unction conti rmedCgobi) { 

" Cgobi. recogRes. text = "yes") 

get„card_info. submit (gen sml O) ; 



h 

// user codes start here 
// 

function handleQ { 

if (field — get_card„info.card_mriiO { 

if Cget_card_info.card_num. length J = 15) { 

prompt. speak C'ajnex should have 15 digits"}; 
get_card_.info.card„num = 

} else 

if (get,card_info*card_num, length 16) { 

prompt . speak C'visa should Tiave IS digits"); 
get_card_info.card_num ^ ""; 

} } 
function gensml O { 

Str = ' <:smlxcredit_card type=" r ; 
str +- card_type.v<tl ue; str +■-= ' "xnurobe r> r ; 
str card_number. value; str += '</numberxexpire> B ; 
str += expiry_date. value; str +« 
1 </expi^e></c^edit_^:ard></sm^> , ; 
^ return str; 

]]> 

</script> 
</body> 
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SAMPLE ASP+ PAGE 

<S{? Page languages" JscripL' 1 AutoEventWireup^" false" Inherits^'Credit . Transaction" %> 
<html><head> 

<! — &.SPX psge foe both voice-only c multimodsl credit c^ard example — > 
<script> 

function handle C) \ 

if (field — getcaid info . uard_nura| 1 

if {got card^info .card_n ura. length != 15) { 
prompt^ speak ("amex snould have 15 digits"); 
get_ca.rd_iiafo.card num = " IT ; 

} 

} else 

if [get_ceirU_ini:o . cardnunu length != 16] \ 
prompt . speak ("visa shouLd have 15 digits") ; 
(jet_card_inf o . card_num « " " ; 

} 

J 

function gensml ( ) { 

str = '<sml><credit_card><card_type>'; 

str +- card_typ$.valae; str +— r </card_Lype><nurnber>' ,- 

str +^ card_immber.value; str -i-- ' </nUmbev-><expi re> T ; 

str += expiry_date. value; str +- ' </expire></credit_card></sjal> * ; 

return str; 

} 

</script> 

<scxipt runat="sc!rver"> 
function Page_Load ( ob j r args) J 
if (PostBack) { 

validator -= new System. Speech . SMLValida tor ( T| . /Cre;diL SDL. Kml ") / 
dsml - validator . Evaluate (args J ; 
Navi gate IChoosePage (dsn 1) ) ; 
} else { 

// initialize fields with args 

} 

) 

</script> 

</head> 

<r;ody> 

<spcech : form id «"ge t_card_inf o" style-"3y3tem_initiative" 

prompt = M . /prompt /get Payment r ' onsubmit=" gensml ( ) "> 
<speech: choice name -="card_ type" pr ompt="WhaL credit card would you use?" 

gramme r-" . /gr_m#card_type5" onPhrase Finish^ "handle ( ) " 

<opt ion>Merican Exp r ess< /opt ion> 

<option>Visa</option> 

<option>Mastercard </option> 
</ speech : ohoi.ee> 

<speech; textbox najme= ,T card_number " prompt="PleasG say the number" 

grammar-". /gramidigits" onPhraseFinish= "handle ( ) "> 
Opeech: Lextbox najoe= ,r expixy_date ,f proiapt-"What is the expiration date? w 

grammar-" . /gramtdates" onPhraseFini sh^ ,T bandle { ] "/> 

</speech: fom> 

</body> 

</html> 
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<html> 
<body> 

<J— the data sec Li on — > 
350 <forrn id="£ et_drink"> 

*v < input nauie~ "drink" /> 

^input type="radio" name="cr8ani'7> 
<input type -""radio" naroe="sugar r V> 
<input name="uid" type="hiddeu''V> 
<^form> 

The speech section — > 
352 cprompt i d= "we 1 came ">Wel cons, caller i </prompt> 

\ <pxpntpt id-"ask"> Do you vant co)ce r coffee, or orange juice? 
</proiapt> 

<prompt id^' r confirm // > I heard <va2\ie href~"drink"/> , Is this 

correct? </ proin.pt> 
<proinpt id="thank:s"> Thank you. Please wait when I get it for 

you, </prompt> 
<prmr.pt id="retry"> Okay, let ' s do this again </prompL> 
<pjrompt id="xeui ompt "> Scxry, I wished that. < /prompt > 
<prorapt id="crearn_sugar ">Do you want cream or sugar with your 
cof fee ?< /prompt > 

<reco id ="reco_drj nk"><gr^rruLiar s tc=" . /drink iypes"/> 
3^.4 364 -<bind tcst="/ f@ confidence 51e? 10]" 



tar get El erne nt=" rep rompt" targetNethod~ "start" 

3vX) targotElemenL-'asr tar getMethod=" start" 

36fi^__ target£lement="xeco drinr'' targetMethod="st3rt "/> 
370^^<bind test ="/drink/cof fee" [(^confidence SgtS 10]" 

targetrJlemeiit^"drink" value=" /drink" 

^ , targetElement=^creain_sugar" targetMethod^'stai t" 

^ '4 taryetElement^^rec^crea^sugar" 

376 " LargetHethod="starL:'7> 

<bind test-"/ [G confidence $gL$ 10]" 

t a i :g e t E .1 ejflen t = " dr ink" valu e= " / d r i n k " 
targetE_Lement= jrr conf irni" targettfethod**" start ** 
targetE2ement="recD_yesno" targetMethod="start"/> 
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<recG id ="reco_creani_sugar"><graminar src=«" . /creaiB+sugar"/> 
<bind test='V[@confidence 10 and 

host (J /gei:_drink/dr in k = Coffee 1 "]" 
356 target Element -"cream" targetAttribure="checkeu'' 

\ value='Vcr earn/ Rvalue" 

targe^El^ment="sugar" targetAttribute="checkeci" 

value^'Ysugar/G value" 
targe tSlement-"con firm" targetMetbod-"start " 
targetElement='Vsco_yesno" targetMethod="start"/> 

</reco> 

<reco id="reco_y2sro ff > <grammar src" w ./yesno"/> 

<l;ind test='7yes ^conf idence $gt£ 10]" 

3 84 . targetSlement==" thanks" targetMethod="srart" 

q jt ~ targetEiement-"get_dr ink" 

3 58 " targetMethod^'submit*- /> 

*V <bind test='7no or ./[ @ confidence 31e$ 10]" 

3 90"^ targetElement- "retry" targetMetbod="sr.art" 

- QO ' target Element- "ask" targetMethod-"scart" 

394—^ targetMethod-"iJtart'V> 

</reco> 

<i-call control section --> 

<smex id-" telephone'' se:"it="start_lisl:ening"><parain 
server="ccx3iUproc"> ... </param> 

<bind targetE.lejnent-"uid" value- ; '7@uid"/> 
360 <bind tS£jt='VCal.l_coiinected r ' 

V 3 61 ' targQtElement="w6l come" targeHMethod^start" 

_ targ^t>:lement = "asJc" targetMethod-"start" 

3 6 2 *"~~ t£rgetElement-"reco_drink" 
363 -^*^ rargetMethod="5tart'V> 

</body> 
</h^ml> 
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1 . Ti t ie of the Inversion 

MARKUP IANGUAGE EXTENSIONS FOR TvEB ENABLED 

RECOGNITION 

2. Claims 

1- A computer readable medium, having a markup language for 
execution on a client device in a client /server system, the 
markup language comprising an instruction indicating a 
grammar to associa'cc with inpuL entered through the client 
device . 

2. The computer readable medium of claim 1 wherein the 
instruction indicating a grammar provides a reference to a. 
location of the grammar. 

3. The computer readable medium of claim 1 wherein the 
grammar is for one of speech recognition, handwriting 
recognition, gesture recognition and visual recognition. 

4. The computer readable medium of claim 3 wherein the 
instruction indicating a grammar includes a reference to a 
grammar for speech recognition. 

5. The computer readaoie medium of claim 1 wherein the 
markup language includes an instruction for receiving a 
recognition result pertaining to recognized speech and 
associating the result with a data field on the client 
device . 

6. The computer readable medium of claim 8 wherein the 
instruction for receiving a recognition result associates the 
recognition result to a plurality of data fields* 
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I. The computer readable medium of claim 1 wherein the 
markup language includes an instruction to audibly render an 
indication when the speech was not recognized. 

8. The computer readable medium of claim I wherein the 
input comprises a text string. 

9. The computer readable medium of claim 8 wherein the 
markup language includes an instruction for receiving the 
text string from another component. 

10. The computer readable medium of claim 9 wherein markup 
language associates the grammar to process the text string. 

II. A computer readable medium including instructions 
readable by a computer which, when implemented, cause the 
computer to handle information by performing steps 
comprising : 

receiving a markup language page from a web server 

having a field for input data; 
receiving input from a user related to the field; and 
sending data indicative of the input and an indication 

of a grammar for recognition. 

12, The computer readable medium of claim 11 wherein the 
indication provides a reference to locations of the grammars. 

13- The computer readable medium of claim 12 wherein the 
indication includes a reference to a language tor speech 
recognition . 
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14. A method for recognition in a client: /server network, the 
method comprising: 

receiving a markup language page having a field for 
input data on a client device connected to a 
network from a wed server at an address on the 
network different than the client device; 

receiving input from a user related to the field on the 
client device; and 

sending data indicative of the input and an indication 
of a grammar Tor recognition to a recognition 
server at an address on the network different than 
the client device. 

15. The method of claim 14 and further comprising: 
normalizing the data indicative of the input prior to 

sending the data. 

lb. The method of claim 41 and further comprising; 

receiving a recognition result pertaining to recognized 
speech and associating the result with a data field 
on the client device. 

17. A computer readable nediura having a markup language for 
execution on a client device in a client /server system, the 
markup language comprising an instruction indicating an 
object model element having an attribute or a method for 
recogni Lion . 

18- The computer readable medium of claim 17 wherein the 
element is for one of speech recognition, handwriting 
recognition, gesture recognition, DTMF recognition and visual 
recognition . 
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19. The computer readable medium of claim 17 wherein the 
element comprises an indication of a grammar for recognition. 

2D. The computer readable medium ot claim 19 wherein e Lenient 
pertains to receiving and processing a text string like 
recognition . 

21. The computer readable medium of claim 20 and further 
comprising a second object model receiving the text string 
from another component. 

22. A computer readable medium having a markup language for 
execution on a cliem: device in a client/server system, the 
markup language comprising an instruction indicating an 
object model element having an attribute or a method for 
audibly prompting. 

23. The computer readable medium of claim 22 wherein the 
markup language comprises one of HTML, XHTML, cHTML, KML and 
WML. 

24. The computer readable medium of claim 22 wherein the 
markup language comprises a scripting language. 

25. The computer readable medium of claim 22 wherein the 
markup language comprises a synchronized markup language. 
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3. Detailed Description of the Invention 
FIELD OF THE INVENTION 

The present invention relate, to access of information 
cv , r a wide area network such as the internet. More 
particularly, the present invention relates f.o web enabled 
recognition allowing information and control on a client side 
to be entered using a variety of methods. 

BACKGROUND OF Till" INVENTION 

Small computing devices such as personal information 
managers (PIM) , devices and portable phones are used with 
ever increasing freguency by people in their day-to-day 
activities. WiLh the increase in processing power new 
available for microprocessors used to run these devices., the 
functionality of these devices are increasing, and in some 
cases, merging. For instance, many portable phones now can be 
used to access and browse the Internet as well as can be used 
to store personal information such as addresses, phone 

numbers and the like. 

in view tha:. these computing devices are being used for 
browsing the Internet, or are used in other server/client 
architectures, it is therefore necessary to enter information 
into the computing device. Unfortunately, due to the desire 
to keep these devices as small' as possible in order -hat they 
are easily carried, conventional keyboards having all the 
letters of the alphabet as isolated buttons are usua: ly not 
possible due to the limited surface area availabl.2 on the 
housings of the computing devices. 

Recently, voice portals such as through the use of 
VcdceXML (voice extensible markup language) have been 
advanced to allow Internet content to be accessed using only 
a telephone. In this architecture, a document server (for 
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example, a web server) processes requests from a client 
through a VoiceXML interpreter. The web server can produce 
VoiceXML documents in reply, which are processed by the 
VoiceXML interpreter and rendered audibly to the user. Using 
voice commands through voice recognition, the user can 
navigate the web. 

VoiceXML is a markup language with flaw control tags; 
however, flow control, does not follow the HTML (Hyper Text 
Markup Lanquage) flow control model, which includes eventing 
and separate scripts. Rather, VoiceXML generally includes a 
form interpretation algorithm that is particularly suited for 
t elephone-based voice-only interaction, and commonly, where 
the information obtained from the user is under the control 
of the system or application. Incorporation of VoiceXML 
directly into applications available in a dient-server 
relationship where graphically user interfaces are also 
provided will require the developer to master two forms of 
web authoring, one for VoiceXML and "he other using HTML (or 
the like), each one following a different flow control model. 

There is thus an ongoing need to improve upon the 
architecture and methods used to provide speech recognition 
in a server/client architecture such as the Internet. The 
authoring tool for speech recognition should be easily 
adaptable to small computing devices such as PIM3, telephones 
and the like. An architecture or method of web authoring that 
addresses one, several or all of the foregoing disadvantages 
is particularly needed. 
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SUMMARY OF THli INVENTION 

A markup language for execution on a client device in a 
client/server system includes an instruction indicating a 
grammar to associate with input data entered through the 
client device. 

With the availability of this extension and as another 
aspect of the present invention, a client device can execute 
instructions to receive a markup language page from a web 
server having a field for input data. The client device can 
then receive input data from a user related to the field and 
send the data and an indication o^: the grammar for 
recognition to a recognition server, typically, located at a 
remote location for processing. 

The recognition server can execute instructions to 
receive the input data and the indication of the grammar to 
perform recognition. The results of recognition can then be 
sent back to the client device or web server for further 
processinq . 



DETAILED DESCRIPTION OF 
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Before describing an architecture of web based 
recognition and methods for implementing the same, it may be 
useful to describe generally computing devices that can 
function in the architecture. Referring cow to FIG. I, an 
exemplary form of a data management device [PIM, PDA or the 
like) is illustrated at 30. However, it is contemplated that 
the presen- invention can also be practiced using other 
computing devices discussed below r and in particular, those 
computing devices having limited surface areas for input 
buttons or the like. For example, phones arid/or data 
management devices will also benefit from the present 
invention. Such devices will have an enhanced utility 
compared to existing portable personal information management 
devices and other portable electronic devices, and the 
functions and compact size of such devices will more likely 
encourage the user to carry the device at all times. 
Accordingly, it is not intended that the scope of the 
architecture herein described be limited by the disclosure of 
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an exemplary data management or FIM device , phone or computer 
herein illustrated. 

An exemplary form of a data management mobile device 30 
is illustrated in FIG. 1. The mobile device 30 includes a 
housing 32 and has an user interface including a display 34, 
which uses a contact sensitive display screen in conjunction 
with a stylus 33. The stylus 33 is used to press or contact 
the display 34 at designated coordinates to select a field, 
to selectively move a starting position of a cursor, or to 
otherwise provide command information such as through 
gestures or handwriting. Alternatively, or in addition, one 
or more buttons 3o can be included on the device 30 for 
navigation, [n addition, other input mechanisms such as 
rotatable wheels, rollers or the like can also be provided. 
However, it should be noted that the invention is not 
intended to be limited by these forms of input mechanisms. 
For instance, another form ot input can include a visual 
input such as through computer vision. 

Referring now to FIG. 2, a block diagram illustrates the 
functional components comprising the mobile device 30. A 
central processing unit (CPU) 50 implements the software 
control functions. CPU 50 is coupled to display 34 so that 
text and graphic icons generated in accordance with the 
controlling software appear on the display 34. A speaker 43 
can be coupled to CPU 50 typically with a digit al-to-analoc 
converter 59 to provide an audible output. Data that is 
downloaded or entered by the user into the mobile device 3 0 
is stored in a non-volatile read/write random access memory 
store 54 bi-directionally coupled to the CPU 50. Random 
access memory (RAM] 54 provides volatile storage for 
instructions that die executed by CPU 50, and storage for 
temporary data, such as register values. Default values for 
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conf igura Lion options and other variables are stored in a 
read only memory (RON) 58. ROM 58 can also be used to store 
the operating system software for the device that controls 
the basic functionality of the mobi.le 30 and other operating 
system kernel functions (e.g., the loading of software 
components into RAM 54) . 

RAM 54 also serves as a sLoiage for the code in the 
manner analogous to the function of a hard drive on a PC Lhat 
is used to store application programs, )'.t should be noted 
that although non -voldLile memory is used for storing the 
code, it alternatively can be stored in volatile memory than 
is not used for execution of the code. 

Wireless signals can be transmit ted/received by the 
mobile device through a wireless transceiver 52, which is 
coupled to CPU 50. An optional communication interface 60 can 
also be provided for downloading data directly from a 
computer (e.g., desktop computer), or from a wired network, 
:.f desired. Accordingly, interface 60 can comprise various 
forms of communication devices, for example, an infrared 
link, modem, a network card, or the like. 

Mobile device 3D includes a microphone 2 9 r ana analog- 
to-digital (A/])] converter 37, and an optional recognition 
program (speech, DTMF, handwriting, gesture or computer 
vision) stored in store 54. By way of example, in response to 
audible information, instructions or commands from a user or 
device 33, microphone 29 provides speech signals , which are 
digitized by A/D converter 37. The speech recognition program 
can perform normalization and/or feature extraction functions 
on uhe digitized speech signals to obtain intermediate speech 
recognition results. Using wireless transceiver 52 or 
communication interface 60 , speech data is transmitted to a 
remote recognition server 204 discussed below and illustrated 
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in the architecture of FIG . 5. Recognition results are then 
returned to mobile device 30 for rendering (e-q- visual 
and/or audible) thereon, and eventual transmission to a web 
server 202 (FIG. 5), wherein the web server 2C2 and mobile 
device 3C operate in a client /server relu Licnship . Similar 
processing can be used for other forms of input, ^or example, 
handwriting input can be digrcized with or without pre- 
processing on device 30, Like the speech data, this form of 
input can be transmitted to the recognition server 204 for 
recognition wherein the recognition results are returned to 
at least one of the device 30 and/or web server 2 02. 
Likewise, DTMF data, gesrure data and visual data can be 
processed similarly. Depending on the form of input, device 
30 (and the other forms of clients discussed below} would 
include necessary hardware such as a camera for visual input. 

FIG. 3 is a plan view of an exemplary embodiment of a 
portable phone 80. The phone 80 includes a display 8 2 and a 
keypad 84. Generally, the block diagram of FIG. 2 applies co 
the phone of FIG. 3, although additional circuitry necessary 
to perform other functions may be required. For instance, a 
transceiver necessary to operate as a phono will be required 
for the* embodiment of FIG. 2; however, such circuitry is not 
pertinent to the present invention. 

In addition to the portable or mobile computing devices 
described above, it. should also be understood thai: the 
present invention can be used with numerous other computing 
devices such as a general desktop computer. For instance , the 
present invention will allow a user with limited physical 
abilities to input or enter text into a computer or other 
computing device when other conventional input devices, such 
as a full alpha-numeric keyboard, ^re too difficult to 
operate . 
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The invention is also operational with numerous other 
general purpose or special purpose computing systems, 
environments or configurations. Examples of well known 
computing systems, environments, and/or configurations that 
may be suitable for use with the invention include, but are 
not limited to, regular telephones (without any screen] 
personal computers, server computers, hand-held or laptop 
devices , multiprocRssor systems, microprocessor-based 
systems, set top boxes, programmable consumer electronics, 
network PCs, minicomputers, mainframe computers, distributed 
computing environments that include any of the above systems 
or devices, and the like. 

The following is a brief description o \=. a general 
purpose computer 12C illustrated in FIG. 4. However, the 
computer 120 is again only one example of a suitable 
computing environment and is not intended to suggest any 
limitation as to the scope of use or functionality of the 
invention. Neither should the computer 120 be interpreted as 
having any dependency or requirement relating to any one or 
combination of components illustrated therein. 

The invention may be described in the general 
context of comput er -executable instructions, such as 
program modules,, being executed by a computer. Generally, 
program modules include routines, programs, objects, 
components, data structures, etc. that perform particular 
tasks or implement particular abstract data types. The 
invention may also be practiced in distributed computing 
environments where tasks are performed by remote processing 
devices that are linked through a communications network. 
In a distributed computing environment, program modules may 
be located in both local and remote computer storage media 
including memory storage devices . Tasks performed by the 
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programs and modules are described below and with the aic 
of figures. Those skilled in the art can implement the 
description and figures as processor executable 
instructions, which car be written on any form of a 
computer readable medium. 

With reference to FCG. 1 , components of computer 
120 may include, but are not limited to, a processing unit 
140, a system memory 150, and a system bus 141 that couples 
various system components including the system memory to 
the processing unir 140. The system bus 141 may be any of 
several types of bus structures including a memory bus or 
memory controller, a peripheral bus, and a local, bus using 
any o± a variety of bus architectures. By way of example, 
and not limitation,, such architectures include Cndustry 
Standard Architecture [ ISA) bus, Universal Serial Bus 
(USB) r Micro Channel Architecture (MCA) bus, Enhanced ISA 
(EISA) bus, Video Electronics Standards Association (VESA) 
local bus, and Peripheral Component Interconnect (PCI) bus 
also known as Mezzanine bus. Computer 120 typically 
includes a variety of computer readable mediums. Computer 
readable mediums can be any available media that can be 
accessed by computer 120 and includes both volatile and 
nonvolatile media, removable and non- removable media. By 
way of example, and not limitation, computer readable 
mediums may comprise computer storage media and 
communication media. Computer storage media includes both 
volatile and nonvolatile, removable and non-removable media 
implemented in any method or technology for storage of 
information such as computer readable ins truotions r data 
structures, program modules or other data. Computer storage 
media includes, but is nor limited to, RAM, ROM, EE PROM, 
flash memory or other memory technology, CD-ROM, digital 
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versatile disks (DVD) cr other optical disk storage, 
magnetic cassettes, magnetic tape, magnetic disk storage or 
other magnetic storage devices r or any other medium which 
can be used to store the desired information and which can 
be accessed by computer 120. 

Communication media typically embodies computer 
readable instructions, data structures, program modules or 
other data in a modulated data signal such as a carrier 
wave or other transport mechanism and includes any 
information delivery media- The term "modulated data 

signal" means a signal that has one or more of its 
characteristics set or changed in such a manner as to 
encode information in the signal. 13y way of example, and 
not limitation., communication media includes wired media 
such as a wired network or direct-wired connection, and 
wireless media such as acoustic, FR, infrared and other 
wireless media. Combinations of any of the above should 
also be included within the scope of computer readable 
media . 

The system memory 150 includes computer storage 
media in the form of. volatile and/or nonvolatile memory 
such as read only memory (RUM) 151 and random access memory 
(RAM) 152. A basic input/output system 1.53 (BIOS), 

containing the basic routines that help to transfer 
information between elements within computer 120 , such as 
during start-up, is typically stored in ROM 151. RAM 152 
typically contains data and/or program modules that are 
immediately accessible to and/or presently being operated 
on by processing unit 140. By way of example, and not 
limitation, FIG . 4 illustrates operating system 54, 
application programs 155, other program modules 156, and 
program data 157. 
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The computer 120 may also include other 
removable /non-removable vola tile/nonvolatile computer 

storage media. By way of example only, H"IG. 4 illustrates 
a hard di.sk drive 161 that reads from or writes to nan- 
removable, nonvolatile magnetic media, a magnetic disk 
drive 171 that reads from or writes to a removable, 
nonvolatile magnetic disk 172, and an optical disk drive 
175 that reads from or writes Lo a removable, nonvolatile 
optical disk 175 such as a CD ROM or other optical media. 
Other reraovable/non-removab Le, volatile/nonvolatile 

computer storage media that can be used in the exemplary 
operating environment include, but are not limited to, 
magnetic tape cassettes, Hash memory cards, digital 
versatile disks, digital video tape, solid state RAM, solid 
state ROM, and the like. The hare disk drive 161 is 
typically connected to the system bus 141 through a non- 
removable memory interface such as interface 160, and 
magnetic disk drive 171 and optical disk drive 175 are 
typically connected to the system bus 141 by a removable 
memory interface, such as interface 170. 

The drives and their associated computer storage 
media discussed above and illustrated in FIG, 4, provide 
storage oC computer readable instructions, data structures, 
program modules and other data for the computer 120. In 
FIG. 4, for example, hard disk drive 161 is illustrated as 
storing operating system 164, application programs 165, 
other program modules 166, and program data 167, Note thai 
these components can either be the same as or different 
from operating system 154, application programs 155, other 
program modules 156, and program data 157. Operating system 
164, application programs 165, other program modules 166, 
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and program ddta 167 are given different numbers here to 
illustrate that, at a minimum, they are different copies. 

A user may enter commands and information into 
the computer 120 through input devices such as a keyboard 
132, a microphone 183, and a pointing device 181 , such as a 
mouse, trackball or touch pad. Other input devices (not 
shown) may include a joystick, game pad, satellite dish, 
scanner, or Lhc like. These and other input devices are 
often connected be the processing unit 140 through a uner 
input interface 180 that is coupled to the system bus, but 
may be connected by other interface and bus structures, 
such as a parallel port, game port or a universal serial 
bus (USB) . A monitor 184 or other type of display device 
is also connected Uo the system bus 141 via an interface, 
such as a video interface 185. In addition to the monitor, 
computers may also include other peripheral output devices 
such as speakers 187 and printer 186, which may be 
connected through an output peripheral interface 188. 

The computer 12 0 may operate in a networked 
environment using logical connections to one or more remote 
computers, such as a remote computer 194. The remote 
computer 194 may be a personal computer, a hand-held 
device, a server, a router, a neliwork PC, a peer device or 
other common network node, and typically includes many or 
all of the elements described above relative to the 
computer 120. The logical connections depicted in FIG. 4 
include a local area network (LAN) 191 and a wide area 
network (WAN) 193, but may also include other networks. 
Such networking environments are commonplace in offices, 
enterprise-wide computer networks, intranets and the 
Internet . 
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When used in a LAN networking environment, the 
computer 120 is connected to the LAN 191 through a network 
interface or adapter 190. When used in a WAN networking 
environment, the computer 120 typically includes a modem 
192 or other means tor establishing communications over the 
WAN 193, such as the Internet. The modern 192, which may be 
in-ernal or external, may be connected to the system bus 
141 via the user input interface 180 r or other appropriate 
mechanism. In a networked environment, program modules 
depicted relative to the computer 120, or portions thereof, 
may be stored in the remote memory storage device. By way 
of example, and not limitation, FIG. 4 illustrates remote 
application programs 195 as residing on remote computer 
194. It will be appreciated that the network connections 
shown are exemplary and other means of establishing a 
communications link between the computers may be used. 

FIG. 5 illustrates architecture 200 lor web based 
recognition as can be embodied in the present invention. 
Generally, information stored in a web server 202 can be 
am^fised through mobile device 30 (which herein also 
represents other forms of computing devices having a 
display screen, a microphone, a camera, a touch sensitive 
panel, etc., as required based on the form of input), or 
through phone 3 0 wherein information is requested audibly 
or through tones generated by phone 80 in response to keys 
depressed and wherein information from web server 202 is 
provided only audibly back to the user. 

More importantly though, architecture 200 is 
unified in that whether information is obtained through 
device 30 or phone 80 using speech recognition, a single 
recognition server 204 can support cither mode of 
operation, fn addition, architecture 200 operates using an 
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extension of. well-known mark-up languagas (e.g. HTML , 
XHTML, cHTML, XML, WML, and the like) . Thus, Information 
stored on web server 202 can also be accessed using well- 
known GUT me- hods found in these mark-up languages. By 
using an extension of well-known mark-up languages, 
authoring on the web server 202 is easier, and leqacy 
applications currently existing can be also easily modified 
to include voice recognition. 

Generally, dsvi cn 10 executes HTML pages, 
scripts, or the like, provided by web server 202. When 
voice recognition is required, by way of example, speech 
data, which can be digitized audio signals or speech 
features wherein the audio signals have been preprocessed 
by device 30 as discussed above, are provided to 
recognition server 204 with an indication of a grammar or 
language model to use during speech recognition. The 
implementation of the recognition server 204 can take many 
forms, one of which is illustrated, but generally includes 
a recognizer 211. The results of recognition are provided 
back to device 30 for local rendering if desired or 
appropriate. Upon compilation of information through 
recognition and any graphical user interface if used, 
device 30 sends the information to web server 2 02 f!oi 
further processing and receipt of further HTML 
pages/scripts T if necessary. 

As illustrated in PIC . 5, device 30, web server 
202 and recognition server 204 are commonly connected, and 
separately addressable, through a network 205, herein a 
wide area network such as the Internee. It therefore is not 
necessary that any of these devices be physically located 
adjacent each other. In particular, it is not necessary 
that web server 202 includes recognition server 204. In 
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this manner, authoring at web server 202 can be focused on 
the application to which it is intended without the authors 
needing to know the intricacies of recognition server 204, 
Rather, recognition server 204 can be independently 
designed and connected to the network 205, and thereby, be 
updated and improved without further changes requirec at 
web server 202. As discussed below, web server 202 can also 
include an authoring mechanism that can dynamically 
generate client-side markups and scripts. In a further 
embodiment, the web server 202, recognition server 204 and 
client 30 may be combined depending on che capabilities of 
the implementing machines. For instance, if the client 
comprises a general purpose computer, e.g. a personal 
computer, the client iaay include the recognition server 
204. Likewise, if desired, the web server 202 and 
recognition server 204 can be incorporated into a single 
machine . 

An aspect o: the present invention is a method 
for processing input data in a client/server system that 
includes receiving from a server a markup language page 
having extensions configured to obtain input data from a 
user of a client device; executing the markup language page 
on the client device; transmitting input data (indicative 
of speech, DiMF, handwriting, gestures or images obtained 
from the user) and an associated grammar to a recognition 
server remote from the client; and receiving a recognitj on 
result from the recognition server at the client. Another 
aspect is a computer readable medium having a markup 
language for execution on a client device in a 
client /server system, the markup language having an 
instruction indicating a grammar to associate with input 
data entered through the client device. 
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Access to web server 202 through phone 80 
includes connection of phone 80 to a wired or wireless 
telephone network 208, that in turn, connects phone 80 to a 
third party gateway 210. Gateway 210 connects phone 80 to a 
lelephony voice browser 212. Telephony voice browser 212 
includes a media server ?_14 that provides a telephony 
interface and a voice browser 216, Like device 30, 
relephony voice browser 212 receives HTML pages /sen pts or 
rhe like from web server 202. More importantly though, the 
HTML pages /scripts are of the form similar to HTML 
pages/scripts provided to device 30- In this manner, web 
server 2 02 need not support device 30 and phone 8 0 
separately, or even support standard GUI clients 
separately. Rather, a common mark-up language can be used. 
In addition, like device 30, voice recognition from audible 
signals transmitted by phone 80 are provided from voice 
browser 216 to recognition server 204, either through the 
network 205, or through a dedicated line 207, for example, 
using TCP/IP. Web server 202, recognition server 204 and 
telephone voice browser 212 can be embodied in any suitable 
computing environment such as the general purpose desktop 
computer illustrated in FIG. 4. 

However, it should be noted that if DTMF 
recognition is employed, this form of recognition would 
generally be performed at the media server 214, rather than 
at the recognition server 204 . In other words, the DTMF 
grammar would be used by the media server. 

As indicated above, one aspect of the present 
invention includes extension of mark -up languages such as 
HTML , XHTML cHTML r XML, WML or with any other SGML-derived 
markup to include controls and/or objects that provide 
recognition in a cl lent /server architecture. In this 
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manner, authors can leverage all the tools and expertise xn 
these mark-up languages that are the predominant web 
development platform used in such architectures. 

Generally, controls and/or objects can include 
one or more of the following functions: recognizer controls 
and/or objects for recognizer configuration, recognizer 
execution and/or post-processing; synthesizer controls 
and/or objects for synthesizer configuration and prompt 
playing; grammar controls and/or objects for specifying 
input grammar resources; and/or binding controls and/or 
objects for processing recognition results. The extensions 
are designed to be a lightweight markup layer, which adds 
the power of an audible, visual, handwriting, etc, 
interface to existing markup languages. As such, the 
extensions can remain independent of: the high-level page 
in which they are contained, e.g. HTML; che low -level 
formats which the extensions used to refer Lo linguistic 
resources, e.g. 'che text-to-speech and grammar formats; and 
the individual properties of the recognition and speech 
synthesis platforms used in the recognition server 204. 

Before describing mark-up languages having 
controls and/or objects suited for recognition , it may be 
helpful to examine a simple GUI example herein embodied^ 
with the HTML mark-up language. Referring to FIG. 6, a 
simple GUI interface comprises submission of credit card 
information to the web server tc complete an on-line sale. 
In this example, the credit card information includes a 
field 250 for entry of the type of credit card being used, 
for example, Visa, MasterCard or American Express. A second 
field 252 allows entry of the credit card number, while a 
third field 254 allows entry of the expiration dato. Submit 
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button 264 is provided to transmit the information entered 
in fields 250, 252 and 254. 

FIG. 7 illustrates the HTML code for obtaining 
the foregoing credit card information from the client. 
Generally, as is common in these forms of mark-up 
languages, the code includes a body portion 260 and a 
script portion 2 62. The body portion 260 includes lines of 
codG indicating the type of action to be performed, the 
form to use, the various fields of information 250, 252 and 
254, as well as a code for submic button 264 (FIG. 6). This 
example also illustrates eventing support and embedded 
script hosting; wherein upon activation of the submit 
button 2 64, a function * verify" is called or executed in 
script portion 262. The w verify" function ascertains 
whether the card number length for each of the credit cards 
(Visa, MasterCard and American Express) is cf the proper 
length . 

FIG. 3 illustrates a client markup that generates 
the same GUI of FIG . 6 for obtaining credit card 
information to be provided to web server 2 04 using speech 
recognition. Although speech recognition will be discussed 
De l ow w ith respect to FIGS. 8-14, it should be understood 
that the techniques described can be similarly applied in 
handwriting recognition, gesture recognition and image 
recognition . 

Generally, the extensions (also commonly known as 
"tags") are a small see of XML elements, wiuh associated 
attributes and DOM object properties, events and methods, 
which may be used in conjunction with a source markup 
document to apply a recognition interface, DTMF or call 
control tu a source page. The extensions formalities and 
semantics are independent of the nature of the source 
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document, so the extensions can be used equally effectively 
within HTML, XHTML, cHTML, XML, WML, or with any ozhor 
SGML-derived markup. The extension follow the document 
object model wherein new functional objects or elements, 
which can be hierarchical, are provided. Each of the 
elements are discussed in derail in the Appendix, but 
generally the elements can include attributes, properties, 
methods, events and/or other "child" elements. 

At this point, it should also be noted that the 
extensions may be interpreted in two different " x modes" 
according to the capabilities of the device upon which the 
browser is being executed on. In a first mode, "object 
mode", the full capabilities are available. The 
programmatic manipulation of the extensions by an 
application is performed by whatever mechanisms are enabled 
by the browser on the device, e.g. a JScript interpreter in 
an XHTML browser, or a WMLScript interpreter in a WML 
browser. For this reason, only a small set of core 
properties and methods of the extensions need to be 
defined, and these manipulated by whatever programmatic 
mechanisms exist on the device or client side. The object 
mode provides eventing and scripting and can offer greater 
functionality to give the dialog author i a much finer 
client -side control over speech interactions. As used 
herein, a browser that supports full event and scripting is 
called an "uplevel browser" . This form of a browser will 
support all the attributes, properties, methods and events 
of the extensions. Uplevel browsers are commonly found on 
devices with greater processing capabilities. 

The extensions can also be supported in a 
w declarative mode". As used herein, a browser operating in 
a declarative mcde is called a v downlevel browser" and does 
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not support full eventing and scripting capabilities. 
Rather, this form of browser will support the declarative 
aspects of a given extension (i.e. the core element and 
attributes), but possibly not all the DOM (document object 
model) object properties, methods and events. This mode 
employs exclusively declarative synLax, and may further be 
used in conjunction with declarative multimedia 
synchronization and coordination mechanisms (synchronized 
markup language) such as SMIL (Synchronized Multimedia 
Integration Language) 2.0. Downlevel browsers will 
typically be found on devices with limited processing 
capabilit ies . 

At this point though, a particular mode of entry 
should be discussed. In particular, use of speech 
recognition in conjunction with at least a display and, in 
a further embodiment, a pointing device as well tc indicate 
the fields for data entry is particularly useful. 
Specifically, in this mode of data entry, the user is 
generally under control of when to select a field and 
provide corresponding information. For instance, in the 
example of FIG . 6, a user could first decide to enter, the 
credit card number in field 252 and then enter the type of 
credit card in field 250 followed by the expiration date in 
field 254. Likewise, the user could return back to field 
252 and correct an errant entry, if desired. When combined 
with speech recognition as described below, an easy and 
natural form of navigation is provided. As used herein, 
this form of entry using both a screen display allowing 
free form selection of fields and voice recognition is 
called 'multi-modal" . 

Referring back to FIG. B, an example of HTML 
mark-up language code is illustrated. Like the HTML code 
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illustrated in FIG • 7, this code also includes a body 
portion 270 and a script portion 212. Also like the code 
illustrated in FIG . 7, the code illustrated in FIG. 8 
includes indications as to the type of action to perform as 
well as the location of the form. Entry of information in 
each of the fields 250 , 252 and 254 is controlled or 
executed by code portions 28 0, 2 82 and 2 84, respectively. 
Referring first to code porLion 280 r on selection of field 
250, for example, by use of stylus 33 of device 30, the 
event "onclick" is initiated which calls or executes 
function u talk" in script portion 272. This action 
activates a grammar used for speech recognition that is 
associated with the type of data generally expected in 
field 2 50. This type of interactlon r which involves more 
than one technique of input (e.g. voice and pen- 
click/roller) is referred as "multimodal" . 

It should be noted that the speech recognition 
extensions exemplified in Fig. 8 are not intended to have a 
default visual representation on the browser of the client, 
since for many applications it is assumed that the author 
will signal the speech enablement of the various components 
cf the page by using application-specification graphical 
mechanisms in the source page. Nevertheless, if visual 
representations are desired, the extensions can so be 
modified. 

Referring now back to the grammar, the grammar is 
a syntactic grammar such as but not limited to a context- 
free grammar, a N-grammar or a hybrid grammar. (Of course, 
DTMF grammars, handwriting grammars, gesture grammars and 
image grammars would be used when corresponding forms of 
recognition are employed. As used herein, a "grammar" 
includes information for performing recognition, and in a 
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further embodiment , information corresponding to expected 
input to be entered, for example, in a specific field) A 
new control 290 (herein identified as "reco"), comprising a 
first extension of the mark-up language, includes various 
elements , two of which are illustrated, namely a grammar 
element "grammar" and a "bind" element. Generally, like the 
code downloaded to a client Irom web server 202, the 
grammars can originate at web server 202 and be? downloaded 
to the client and/or forwarded to a remote server for 
speech processing. The grammars can then be stored locally 
thereon in a cacha. Eventually, the grammars are provided 
to :he recognition server 204 for use in recognition. The 
grammar element is used to specify grammars, either inline 
or referenced using an attribute. 

Upon receipt of recognition results from 
recognition server 204 corresponding to the recognized 
speech, handwriting, gesture, image, etc., syntax of reco 
control 290 is provided to receive the corresponding 
results and associate it with the corresponding field, 
which can include rendering of the text therein on display 
34. In the illustrated embodiment, upon completion of 
speech recognition with the result sent back to the client, 
it deactivates the reco object and associates the 
recognized text with the corresponding field. Portions 282 
and 28 4 operate similarly wherein unique reco objects and 
grammars are called for each of the fields 252 and 254 and 
upon receipt of the recognized text is associated with each 
of the fields 252 and 254. With respect to receipt of the 
card number field 2 52, the function M handle" checks the 
length of the card number with respect to the card type in 
a manner similar to that described above with respect to 
FIG. 1. 
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Generally, use of speech recognition in 
conjunction with architecture 200 and the client side mark- 
up language occurs as fellows; first, the field that is 
associated with the speech to be given is indicated. In the 
illustrated embodiment f the stylus 33 is used; however, it 
should be understood thai: the present invention is nob 
limited to use o ?. the stylus 33 wherein any form of 
indication can be used such as buttons, a mouse pointer, 
rotatable wheels or the like. Corresponding event such as 
"onClick:" can be provided as is well known with use of 
visual mark-up languages. It should be understood that the 
present invention is not limited to the use of the 
"onCJick" event to indicate the start of voice, 
handwriting, gesture, etc commands. Any available GUI event 
can be used for the same purpose as well, such as 
u onSeloct f . In one embodiment, such eventing is 
particularly useful, for it serves to indicate both the 
beginning and/or end of the corresponding speech, it should 
also be noted that the field for which the speech is 
directed at can be indicated by the user as well as 
programs running on the browser that keep track of user 
interactions . 

At this point, it should be stated that different 
scenarios of speech recognition require different behaviors 
and/or outputs from recognition server 204. Although the 
starting of the recognition process is standard in all 
cases - an explicit start () call from uplevel browsers, or 
a declarative <reco> element in downlevel browsers - the 
means for stopping speech recognition may differ- 
In the example above, a user in a multimodal 
application will control input into the device by, for 
example, tapping and holding on a pressure sensitive 



(17 9 ) )03- 1 3 1 772 ( P 2 0 0 3- 1 3 1 7 7 2 A) 



display. The browsGr then uses a GUI event, e.g. "pen-up", 
to control when recognition should stop and then returns 
the corresponding results. However, in a voice-only 
scenario such as in a telephone application (discussed 
below) ur in a hands-tree application, the user has no 
direct control over rhe browser, and the recognition server 
204 or the client 30 P must cake the responsibility of 
deciding when to stop recognition and return the results 
(tyoically once a path through the grammar has been 
recognized) . Further, dictation and other scenarios where 
intermediate results need to be returned before recognition 
is stopped (also known as "open microphone") not only 
requires an expiicio stop function, but also needs to 
return multiple recognition results to the client 30 and/or 
web server 202 before the recognition process is stopped. 

In one embodiment, the Reco element can include a 
"mode" attribute to distinguish the following three modes 
of recognition, which instruct the recognition server 204 
how and when to return results. The return of results 
implies providing the "cnRecc" event or activating the 
"bind" elements as appropriate. In one embodiment, if the 
mode is unspecified, the default recognition mode can be 
" automatic" . 

h'IG. 12 is a pictorial representation of 
operation of the v automatic" mode for speech recognition 
(similar modes, events, etc. can be provided for other 
forms of recognition) . A timeline 281 indicates when the 
recognition server 204 is directed to begin recognition at 
283, and where the recognition server 204 detects speech at 
28b and determines that speech has ended at 28"/. 

Various attributes of the Reco element control 
behavior of the recognition server 204. The attribute 
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"initialTiineout" 28 9 is the time between the start of 
recognition 283 and the detection of speech 285. 11 f this 
time period is exceeded, "onSilence" event 291 will be 
provided from the recognition server 2 04 r signaling that 
recognition has stopped. If the recognition server 204 
finds the utterance to be unrecognizable, an "onNoReco" 
event 293 will be issued, which will also indi.ca.te that 
recognition has stopped. 

Other attributes that can stop or cancel 
recognition include a M babble Timeout" attribute 295 , which 
is the period of time in which the recognition server 204 
must return a result after detection of speech at 28b. If 
exceeded, different events are issued according to whether 
an error has occurred or not. If the recognition server 204 
is still processing audio, for example, in the case of an 
exceptionally long utterance, the " onNoReco" attribute 293 
is issued. However, if the " babble-Timeout" attribute 2 95 is 
exceeded for any other reason, a recognizer error is more 
likely and an " onTimsout" event 297 is issued. Likewise, a 
"maxTimeout" attribute 299 can also be provided and is for 
the period of time between the start of recognition 28 3 and 
the results returned to the client 30. If this time period 
is exceeded, the "onTiraoout' event 297 is issued. 

"f, however, a time period greater than an 
"endSiience" attribute 301 is exceeded, implying that 
recognition is complete, the recognition server 204 
automatically stops recognition and returns its results. It 
should be noted that the recognition server 204 can 
implement a confidence measure to determine if the 
recognition results should be returned. If the confidence 
measure is belcw a threshold, the "criNoReco" attribute 293 
is issued, whereas if the confidence measure is above the 
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threshold a "onNoReco" attribute 3 03 and the results of 
recognition are issued. FIG. 12 thereby illustrates thai: in 
''automatic mode" no explicit stop () calls are made. 

HTG. 13 plctorially illustrates w single mode" 
operation of the recognition server 2C4. Attributes and 
events described above with respect to the tt automatic mcde" 
are applicable and are so indicated with the same reference 
numbers. However r in this mode of operation, a stop () call 
305 is indicated on timeline 281. The stop () call 305 
would correspond to an event such as ^pon-up" by the user. 
In this mode of operation, the return of a recognition 
result is under the control of the explicit stop () call 
305. As with all modes of operation, the v " onSilence" event 
291 is issued if speech is not detected within the 
v initialTimeout" period 289 r but for this mode of cperation 
recognition is not stopped. Similarly, a v> onNoReco" event 
293 generated by an unrecognizable utterance before the 
stop () call 305 does not stop recognition. However, if the 
time periods associated with the "babbleTimeout" attribute 
2 95 or the "maxTimeouL'' attribute 299 are exceeded 
recognition will stop. 

FIG. 14 plctorially illustrates "multiple mode" 
operation of the recognition server 204. As indicated 
above, this mode of operation is used for an ^ open- 
mic^ophone" o~ in a dictation scenario. Generally, in this 
mode of operation, recognition results are returned at 
intervals until an explicit stop ()_ call 305 is received 
or the time periods associated with the %% babbleTimoout" 
attribute 295 or the "max Time out" attribute 299 are 
exceeded. It should be noted, however, that after any 
"onSilence" event 2 91, "anReeo" event 30 3, or " onNoReco" 
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event 293, which does not stop recognition, timers for the 
' v babbleTimeout" and "maxTimeout" periods will be reset. 

Generally, in this node of operation, for each 
phrase that is recognized, a "onReco" event 303 is issued 
and the result is returned until the stop (} call 305 is 
received. If the "onSilence" event 291 is issued due to an 
unrecognizable utterance these events are reported but 
recognition will continue. 

As indicated above, the associated reco object or 
objects for the field is activated, which includes 
providing at least an indication to Lhe recognition server 
204 of which grammar to use. This information can accompany 
the speech data recorded at the client 30 and sent to the 
recognition server 204. As indicated above, speech data can 
comprise streaming data associated with Lhe speech entered 
by the user, or can include pre-processed speech data 
indicating speech features that are used during speech 
recognition. In a further embodiment, client side 
process trig can also include normalisation of the speech 
data such that the speech data received by the recognition 
server 204 is relatively consistent from client to client. 
This simplifies speech processing of the recognition server 
204 thereby allowing easier scalability of the recognition 
server 204 since Lhe recognition server can bo made 
stateless with respect to the type of client and 
communication channel . 

Upon receipt of the recognition result from the 
recognition server 204 f the recognition result is 
associated with the corresponding field, and client -side 
verification or checking can be performed, if desired. Upon 
completion of all of the fields associated with the code 
currently rendered by the client, the information is sent 
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to web server 202 for application processing. From the 
foregoing, it should be clear Lhat although the web server 
202 has providec code or pages/scripts suitable for 
recognition to -he client 30, the recognition services are 
not performed by the web server 202, but rather by the 
recognition server 204. The invention, however, does not 
preclude an implementation where the recognition server 204 
is collocated with the web server 202, or the recognition 
server 204 is pare of the client 30. In other words, the 
extensions provided herein are beneficial even when the 
recognition server 204 is combined with the web server 202 
or client 30 because the extension provide a simple and 
convenient interlace between these components. 

While not shown in the embodiment illustrated in 
FIG . 8, the reco control can also include a remote audio 
object (RAO) to direct the appropriate speech data to the 
recognition server 204. The benefit for making RAO a plug- 
in object is to allow a different one for each different 
device or client because the sound interface may likely be 
different. In addition, the remote audio object can allow 
multiple reco elements to be activated at the same Lime. 

FIGS. 9A and 9B illustrate a voice -only mark-up 
language embodied herein as HTML with pages /scripts . As 
clearly illustrated, the code also Includes a body portion 
300 and a script portion 302. There is another extension of 
the markup language - prompt control 303 which include 
attributes like bargein. However, speech recognition is 
conducted differently in the voice-only embodiment of H^IGS. 
9A and 98. The process Is now controlled entirely by the 
script function "checkFilled" which will determine the 
unfilled fields and activate correspondent prompt and new 
objects. Nevertheless, grammars arc activated using the 
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same context as that described above with respect to FIG. 
8, wherein speech data and the indication of the grammar to 
use are provided to the recognition server 204. Likewise, 
the output received from the recognition server 204 is 
associated with fields of the client (herein telephony 
voice browser 212} . 

Other features generally unique to voice-only 
applications is an indication to the user when speech has 
not been recognized. In multimodal applications such as Fig 
8, A onNoReco r simply pnts mill value on the displayed field 
to indicate no -recognition, thus no further action is 
required. In the voice-only embodiment: , " onNoRsco" 305 
calls or executes a function "mumble", which forwards a 
word phrase to recognition server 204, that in turn r is 
converted to speech using a suitable text-to- speech system 
307 (FIG. 5). Recognition server 204 returns an audio 
stream to the telephony voice browser 212, which in turn, 
is transmitted to phone 80 to be heard by the user, 
likewise, other waveform prompts embodied in the voice-only 
application are also converted, when necessary, to an audio 
stream by recognition server 204. 

It should be noted that in this example after 
playing the welcome prompt via function "welcome", function 
" checkFilled" prompts one user for each of the fields and 
activates the appropriate grammars, including repeating the 
fields that have been entered and confirming that the 
information is correct:, which includes activation of: a 
vv confirmation" grammar. Note in this embodiment, each of 
the reco controls is initiated from the scrip, portion 30?, 
rather than the body portion or: the previous example. 

As another aspect o± the present invention, the 
markup language executable on different types of client 
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devices (e.g. multimodal and non-display f voice input based 
client devices such as a telephone) unifies at least one of 
speech-related events, GUI events and telephony events for a 
web server interacting with each of the client devices. This 
is particular advantageous for it allows significant portions 
of the web server application to be written generically or 
independent of the type of client device. An example is 
illustrated in FIGS. 8 and 9A r 9B with the "handle" 
functions , 

Although not shown in Fig 9, there are two more 
extensions to the markup language to support telephony 
functionality - DTMF (Dual Tone Modulated Frequency) 
control and call control elements or objects. UTMF works 
similarly to reco control. It specifies a simple grammar 
mapping firom keypad string to text Input- For example, " V 
means grocery department, " 2" mean pharmacy department, 
etc. On the other hand, call object deals with telephony 
functions, like call transfer and 3 rd party call. The 
attributes, properties, methods and events are discussed in 
detail in the Appendix. 

FIGS . 1QA and 10B illustrate yet another example 
of a mark-up language suitable for a voice-only mode of 
operation. In this embodiment , the user is allowed to have 
some control over when information is entered or spoken. In 
other words, although the system may initiate or otherwise 
direct the user to begin speaking, the user may offer more 
information than what was initially asked for. This is an 
example of "mixed initiative". Generally, in this form of 
dialog interaction^ the user is permitted to share the 
dialog initiative with the system. Besides the example 
indicated above and discussed below in deLail where the 
user provides more information then requested by a prompt, 



(S6))03-131772 (P2003-131772A) 



the user could also switch tasks when not prompted to do 
so. 

In the example of FIGS 10A and 10B, a grammar 
identified as "do_field" includes the information 
associated with the grammars " g _card_types" , XN q_carri_num' r 
and "gexpiry date" . In this example, telephony voice 
browser 212 sends speech data received from phone 80 and an 
indication to use the "do_field" qrammar to recognition 
server 204 upon receipt of the recognized speech as denoted 
by "onReco", che function "handle" is called or executed 
thai include associating the values for any or all of the 
fields recognized from the speech data. Cn other words, the 
result obtained from the recognition server 204 also 
includes indications for each of the fields. This 
information is parsed and associated with the corresponding 
fields according to binding rules specified in 405. As 
indicated in FIG * S, the recognition server 204 can include 
a parser 309. 

From FIGS . 1, 8, 9A, 9B, 10A and 10B r a very 
similar web development framework is used. Data 
presentation is also very similar in each of these cases. 
In addition, the separation of data presentation and flow 
controls allow maximum reusability between different 
applications (system initiative and mixed-initiative), or 
different modalities ( GUI web-based, voice-only and 
multimodal). This also allows a natural extension from 
voice-only operation through a telephone to a multimodal 
operation when phones include displays and functionalities 
similar to device 30. Appendix A provides further details 
of the controls and objects discussed above. 

As Indicated above, uplevel browsers can use 
scripting in order to perform various needs such as 
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invoking the function "handle" in the above examples to 
assign the recognition results. In Lhe embodiments 
described above and as further described in Appendix A at 
2.1.2, the "bind" element; will parse the recognition 
results and assign values wherein the "bind" element is a 
subelement or child element of the "reco" element. 

Although scripting can be useful, it is believed 
by many not to be always the best form of browser 
Implementation due, for example, to security concorns . 
Therefore, in yet another embodiment or aspect of the 
present invention, the "bind" element is a high "Level 
element (similar to 

" recu" ) and is provided with other richer properties, which 
can in effect mimic scripting without scripting per se. 

Without using scripting or without using the 
aspect of the present invention discussed below, some of 
the capabilities discussed below such as sophisticated 
dialog effects could only be accomplished by submitting a 
page back to the web server 202 r executing the application 
logic thereon to generate a new page, and sending the page 
back to the client device. This aspect of the present 
invention allows a programmer to invoke methods on objects 
of the page without incurring a server roundtrip. 

In the embodiments described above, the ^bind" 
element has only the attributes %> Targetttlement" and 
"Target Attribute" for assigning recognition results to a 
field in the form or web page. In a further embodiment, the 
"bind" element also includes a " TargetMethod" , which Is 
added for object method Invocation. The use and capability 
of ^TaxgetMethod" is the principal technique for mimicking 
scripting. For example, the following syntax can bo used to 
invoke the "X" method of the object "0BJ1" : 
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<bind TargetElement -= "OBol" TargetMethod 

w X" . . . > - 

Note that although the examples shown here follow 
the HTML /XHTML event syntax, those skilled at art should 
find it straightforward to generalizes the <bind> usage to 
other eventing mechanisms, including but not limited to W3C 
Document Object Model Level 2 or Level 3 eventing standard, 
ECMA Common language Infrastructure (CLI) event model, Java 
programming language event model, W3C Synchronous 
Multimedia Integration Language (SMIL), and the upcoming 
W3C XML Events standard proposal. 

FIGS - 15A and 15B are a page oT mark-up language 
executable on a client, particularly a downlevel browser. 
In this example, the user is asked through audio prompts 
for a drink. "he system then confirms what drink was 
ordered. Depending on the recognition results, the v bind" 
element guides the execution using declared logic. When the 
drink is confirmed, the form is submitted back to the web 
server 202, all without scripting. 

Generally, the mark-up example of FIGS. ISA and 
15B includes a data section 350, a speech section 352 and 
user interface sections 354, 336 and 358. Section 354 
receives the recognition result from Lhe general inquiry as 
to what drink the user would like and directs interactive 
recognition flow to either roprompting, inquiring as Lo 
whether cream or sugar is needed, or confirming the drink 
ordered. In particular, section 356 receives a recognition 
result when cream or sugar is also ordered. Section 3 58 
receives the recognition result for confirmation of the 
drink. Section 360 is a call control section employing a 
new messaging object * SMEX" r which is discussed further 
below. 
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As indicated above, the n bind" element of this 
aspect of the present invention includes object method 
invocation^ which will initiate u^er interaction in the 
example of FIGS. 15A and 15E by playing the "welcome" 
prompt when the "start" method on Lhe "welcome" object is 
executed at 361. 

The user is then asked M Do you want coke, coffee 
or orange juice?" by executing the "start" method of the 
"asked" object at 3 62. Recognition is then performed by 
invoking the "start" method on the recognition 'ieco^ 
drink" object at 363. 

The mark-up of section 354 is then executed 
wherein the grammar used by the recognition server 204 is 
provided by the Xpach statement M . /drink types". Note that 
although this example utilizes W3C Xpath language, those 
skilled at art should find it straightforward to extend the 
concept to other standard languages r including but not 
limited to W3C XML Query Language (XQL ) . As specified by 
the "bind" element 364, if: the recognition result received 
from the recognition server 204 has a confidence score less 
than 10, Lhe prompt object w repr omp t" is executed at 3GG f 
followed by the prompt object "ask" 368 r at which point, 
the recognition object " reco_drink" is reinitiated at 370. 
If the recognition result returned is "coffee" having a 
confidence greater than 10 r the field w drink" is assigned 
the value of the recognition result at 372 and the user is 
then prompted as to whether or not he/she would like cream 
or sugar by the prompt object " cream_sugar" at 37 4- The 
recognition object 'heco_creani sugar" in section 356 is 
then invoked at 376. Otherwise, if the confidence score of 
the recognition result is greater than 1C, but not coffee, 
the field drink is then again assigned at 378. Conf irmaLion 
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of the recognition result is provided at 380 by executing 
the prompt object "confirm", followed by invocation of the 
recognition object "reco yesno" in section 353 at 382. If 
the user answers w yes" with a confidence score greater than 
10, the prompt object "thanks" is played at 384 and then 
the form is then submitted at 386. Otherwise, if the user 
answers "no" or the confidence score of the recognition 
result is less than 10, the prompt object "retry" is 
executed at 3 90, once again followed by the prompt object: 
"ask" being executed at 392, and invocation of the 
rt reco drink" recognition object at 394. 

From ehe foregoing example, the "bind" element 
allows multiple invocations of methods as indicated in 
sectiuns 354, 356 or 358. If desired, multiple assignments 
of the recognized result can also be declared. In the 
embodiment illustrated, when multiple assignments and 
method invocations are declared, they are executed In the 
document order. 

In a further embodiment, a convention for passing 
method arguments is also provided. In other words, some 
methods may require a list of arguments. This is achieved 
by using "arg" subelement. For example, given the following 
mark -up : 

<bind TargotElement = "03 J" TargetMethod 

"F" ><arg>X</arg><arg>Y</argX/bind> 
is equivalent to * OBJ. F (X, Y)" , or "OBJ" is an object that 
has a method lx F" with parameters or arguments "X" and " Y" . 

The "bind" element can also include an "event/' 
attribute, which declares the bind element is intended for 
which event. For example, the mark-up: 

<bind event = "onNoReco" = TargetElement 

"prompt V s TargetMethod = w start" /> 
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means when the "onNoReco" event is yenl, the method "start" 
of "he object ^promptl" will be invoked. To be consistent 
with use of the "bind" slement as a child element of the 
vl Recr" element as described above for example with respect 
to FIG. 8, the default attribute for the "bind" element is 
v onReco" . 

The "bind" element as a high level element can 
include any of the events specified in section 2.4 oE the 
Appendix. In addition, the "bind" element can also include 
a N> onError" event having a "status" attribute that can be 
accessed and used to direct program flow. To the extent 
other events of the "bind" element 'have w status" 
attributes, these can he accessed as well. 

in addition to checking the conditions of: the 
recognition result, the current document or page being 
executed can also be checked as well- In particular, bcth 
the "test/' and the "value" attribute can be extended to 
include a ''host" primitive that refers to the root node of 
the containing document. For example, referring back to 
FIGS. 15A, 1.5B, the example contained therein has 
additional logic at section 3ti to ask whether the user 
wants cream or sugar when he/she asks for coffee. The flags 
for adding cream or sugar and thus invocation of section 
356, will be turned on only if the drink field is ,x coffee" 
as specified by the markup 

"host () ,/get_drink/drink=' coffee'" - 

It should also be noted, the "bind" element is 
not only applicable to recognition results from the speech 
server 204 and receive or assign values in the document, 
but also to message objects (herein denoted by " smex" r for 
example, from applications running on the client device. In 
the example of FIGS. 15 A and 15B r the page is executed when 
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a telephony application running on the client device 
defects a call. In section 360, the "bind" element executes 
or plays the "welcome" prompt and begins recognition by 
executing the w reco_drink" object, when the message 
" /Call_connected" is received. Like the recognition results 
received from the speech server 204, the messages received 
can vary greatly. Some of the messages are well -defined in 
order to initiate desired program flow. Others can be 
received and p-ocessed (e.g. parsed just like received 
recognition results of the recognition server. For example, 
this allows the markup to be used like a natural language 
parser of text from a keyboard. The reco element in 
Appendix A includes a property for performing this 
function. Likewise, the prompt element can be used to 
provide a textual message for dynamic content or audio wave 
files by using the property " inner text" , also further 
explained in the Appendix A. Eventing can be similar to 
eventing for recognition results. For example, eventing can 
include w onReceived" , which gets ?cr.t when the message 
source (e.g. application running on the client device) has 
a message available for the browser. 

The "smex" or message object thus allows the 
markup tags discussed herein Ho be extended to other 
components or applications running on the client device. As 
anocher example, the message object can be used to 
communicate to a TTY component: for the hearing impaired 
running on the client device. Rather than using speech 
recognition, the TTY component will provide a message of 
whar the user has typed. This message is then used as if a 
recognition result was received from the recognition server 
wherein the message can be parsed and assigned to fields of 
the form, or other processing can take place using the 
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"reco" r "grammar" or "bind" elements described above. 
Further discussion of the message or "smex" object is 
provided in Appendix A. 

The "bind" element can also include a "for" 
attribute, which allows its action to be attached to other 
objects on the page. For example, a mark-up such as: 

<bind for = "prompt 1" event = " onComplet e" 

targetElement = "prompt 2" = targetMethod = 

"s„art" />. 

will invoke the start method of the object "prompt 2" when 
the object "prompt V sends the event " onComplet e" - 

Referring bac^ to FIG. 5, web server 202 can 
include a server side plug-in declarative authoring tool or 
module 320 (e.g. ASP or ASP+ by Microsoft Corporation, JSP, 
or the like) . Server side plug-in module 320 can 
dynamically generate client -side mark-ups and even a 
specific form of mark-up for Lhe type of client accessing 
the web server 202. The client information can be provided 
to the web server 202 upon initial establishment of the 
client/server relationship, or the web server 202 can 
include modules or routines to detect the capabilities of 
the client. In this manner, server side plug -in module 120 
can generate a client side mark-up for each of the voice 
recognition scenarios, i.e. voice only through phone 80 or 
multimodal for device 30. By using a consistent client side 
model (reco and prompt controls that can be used in each 
application) , application authoring for many different 
clients is significantly easier. 

In addition Lc dynamically generating client side 
mark-ups, high-level dialog modules, like getting credit 
card information illustrated in FIG. 6 with a mark-up 
examples of FIGS - 8 r 9A and 9B, can be implemented as a 
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server-side control as stored in store 324 tor use by 
developers in application authoring. In general, the high- 
level dialog modules 324 would generate dynamically client- 
side markup and script in both voice-only and multimodal 
scenarios based on parameters specified by developers. The 
high-level dialog modules can include parameters to 
generate client-side mark-ups to fit the developers ' needs. 
For example, a credit card information module can include a 
parameter indicating what types of credit cards the 
client/side mark-up script should allow. A sample ASP+ page 
using in server side plug-in module 320 is illustrated in 
FIG, 11. 

Although the present invention has been described 
with reference to preferred embodiments, workers skilled in 
the art will recognise that changes may be made in form and 
detail without departing from the spirit and scope of the 
invention . 
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1 Introduction 

The following tags are a. set of markup elements that allows 
a document to use speech as an input cr output medium. The 
tags are designed to be self-contained XML thai: can be 
imbedded into any SGML derived markup languages such as 
HTML, XHTML , cHTML, SMIL, WML and the like. The tags used 
herein are similar to GAPI 5.0, which are known methods 
available from Microsoft Corporation of Rednond, Washington. 
The tags, elements, events, attributes r properties, return 
values, etc. are merely exemplary and should not be 
considered limiting. Although exemplified herein for speech 
and DTMF recognition, similar tags can be provided for other 
forms of recognition. 

The main elements herein discussed are: 

<prompt ...> for speech synthesis configuration and 

prompt playing 

<raco ...> for recognizer configuration and recognition 

execution and post-prcces s ing 

<gramina.r ...> for specify trig Input gramma x resources 

<b±nd ...> for processing ot recognition results 
<dtiaf ...> for configuration and control of DTMF 

The Reco element is used to specify possible user inputs and 
a means for dealing with the inpuc results. As such, its 
ma Ln elements can be <gr arnmar > and <bind>, and i~ contains 
resources for configuring recognizer properties. 
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Reco elements are activated programmatically in uplevel 
browsers via Start and Stop methods, or in SMIL-enabled 
browsers by using SMIL comands . They are considered active 
deelaratively in downlevel browsers (i.e. non script- 
supporting browsers) by thoir presence on the pace. In order 
to permit: ohe activation or multiple grammars in parallel, 
multiple Reco elements may be considered active 
simultaneously. 

Recos may also take a particular mode - 'automatic' r 
* single' or 'multiple' - to distinguish the kind of 
recognition scenarios which they enable and the behavior of 
the recognition platform. 

2.1 R&czo coj7texlt 

The Reco element contains one or more grammars and 
optionally a set of bind elements which inspect Lhe results 
of recognition and copy the relevant portions to values In 
the containing page. 

In uplevel browsers, Reco supports rhe programmatic 
activation and deactivation of individual grammar rules. 
Note also that all top-level rules in a grammar are active 
by default for a recognition context. 

2.1.1 <gxainmar> elsment 

lhe grammar element is used to specify grammars, either 
Inline or referenced using the arc attribute. Tit least one 
grammar (either inline or referenced) is typically 
specified. Inline grammars can be text-based grammar 
formats, while referenced grammars can oe text-based or 
binary type. Multiple grammar elements may be specified. If 
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grammars are added as extra rules within the same grammar. 
Any rules with the same name will be overwritten. 

Attributes : 

* src: Optional if .inline grammar is specified. URI of 
the grammar to be included. Note that all top-level 
rules in a grammar are active by default for a 
recognition context. 

• langID: Optional. String indicating which language 
speech engine should use. The string format follows the 
xmlrlang definition. For example, langl D="en-us " 
denotes US English. This attribute is only effective 
when the langiD is not specified in the grammar URI. Zt 
unspecified, defaults to US English. 

If the LangID is specified in multiple places then 
langID follows a precedence order from the lowest scope 
- remote grammar file (i.e language id is specified 
within the grammar file} followed by grammar element 
followed by reco element. 

<grammar src^ // FromCity.xml ; ' /> 
or 

f 

<gramma r> 

<rule t op level-" act ive"> 
<p>from </p> 

<ruleref name^ : "cit ies" /> 
</rule> 

<rule ndme="cities"> 
<1> 

' <p> Cambridge </p> 
<p> Seattle </p> 
<p> London </p> 
</!> 
</rule> 
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< / gramnar> 

If both a src-ref erenced grammar and an inline grammar are 
specified, the inline rules are added to the referenced 
rules, and any rules with the same name will be overwritten. 

2.1,2 <bind> Glemsnt 

The bind elemenL is used to bind values from the recognition 
results into the page. 



The recognition results consumed by the bind element can be 
an XML document containing a semantic markup language (SML) 
for specifying recognition results. Its contents include 
semantic values, actual words spoken, and confidence scores. 
SML could also include alternate recognition choices {as in 
an N-best recognition result) . A sample SML document for the 
utterance ^I'd like to travel from Seattle to Boston" is 
illustrated below: 

<sml confidence="40"> 

<travel tent-"I r d like to travel from. Seattle to 
Boston "> 

<origin_city conf idence^"45"> Seattle 

</origin_city> 

<desc_city confidence ="35"> Boston 

</dest _city> 

</travel> 
</sml> 

Since an in- grammar recognition is assumed to produce an XML 
document - in semantic: markup language, or SML - the values 
to be bound from the SML document ctre referenced using an 
XPath query- And since the elements in the pa go into which 
the values will be bound should be are uniquely identified 
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(they are likely to be form controls) , these target elements 
are referenced directly. 

Attributes : 

• targetElement: Required. The element to which the value 
content from the SML will be assigned (as in W3C SMIL 
2,0 ) . 

=> tairgestAttr/ibute : Optional. The attribute of the target 
element to which the value content from the SMI will be 
assigned (as with the abtrlbutoNamG attribute in SMIL. 
2.0 ), If unspecified, defaults to "value". 

• test: Optional. An XML Fafcfcegg (as in the W3C XML DOM 
specification) soring indicating the condition under 
which the recognition result will be assigned. Default 
condition is true. 

• value: Required. An XPATH (as in the W3C XML 00M 
specification) string that specifies 'cho value from the 
recognition result document to be assigned to the 
target element. 

Example : 

So given the above SML return, the following reco element 
uses bind to transfer the values in origin_city and 
dest_city into the target page elements txtBoxOrigin and 
OxtBoxDest : 

<input name= T, txtBoxOrlgin fT type=' r texl: , 7? 
<input name="txtBoxDes t" type="t.ext" /> 

<reco id^"t ravel "> 

<grammar src=" . /city. xml" /> 
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<bind targetElement^ rr txtBoHOrigin 1T 

valuG = "//nrigi_n_city" /> 
<bijid targetElement="txtBoxDest " 

value="//dest_city" /> 

</reco> 

This binding may be conditional, as in the following 
example, where a test is rr.ade on the confidence attribute of 
the dest_city result as a pre-condition to the bind 
operation : 

<bind targetElement^'tx'cBoxDest ri 
va 1 ue= f 1 / / de is t_c i t y " 

test=Vsrnl/dest_city [ @ confidence $qt$ AO]' 1 

/> 

The bind element is a simple declarative means cf processing 
recognition re suit a on downlevel or op level browsers. For 
more complex processing, the reco DOM object supported by 
uplevel browsers implements the onReco event handler to 
permit programmatic script analysis and post -processing of 
the recognition return. 

2.2 Attributes and properties 

The following attributes are supported by all browsers, and 
the properties by uplevel browsers. 

2.2.1 Attributes 

The toiiowing attributes of Reco are used to configure the 
speech recognizer; for a dialog turn. 
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• ini-bialTimeout : Optional. The time in milliseconds 
bebween start of recognition and the detection of 
speech. This value is passed to the recognition 
platform, and if exceeded, an onSilence event will 
be provided from the recognition platform (see 
2.4.2). If not specified, the speech platform will 
use a default value. 

• babbleTimeon c : Optional. The period of: time in 
milliseconds in which the recognizer must return a 
result after detection of speech. For recos in 
automatic and single mode, this applies to the 
period between speech detection and the stop call. 
For recos in ''multiple' mode, this timeout applies 
to the period between speech detection and each 
recognition return - i.e. the period is restarted 
alLer each return of results or obher event. If 
exceeded, different events are thrown according to 
whether an error has occurred or not. If the 
recognizer is still processing audio - eg in the 
case of an exceptionally long utterance - the 
onNoReco event is thrown, with status code 13 (see 
2.4.4). It the timeout is exceeded for any other 
reason, however, a recognizer error is more likely, 
and the onTirneout event is thrown. If not specified, 
the speech platform will default to an internal 
value . 

-' maxTiiueout: Optional. The period of time in 

milliseconds between recognition start and results 
returned to the browser. If exceeded, the onTirneout 
event is thrown by the browser this caters for 
network: or recognizer failure in distrrbuted 
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environments. For recos in hnultiple r mode, as with 
babbleTimeout r the period, is restarted after the 
return of each recognition or other event. Note that 
the max Time out atiiribute should be greater than or 
equal to the sum of ini tialTimeout and 
bubbleTimeout . If not specified, the value will bo a 
browser default. 

• endSilence: Optional, For Recoa in automatic mode r 
the period of .silence in milliseconds after the end 
of an utterance which must be free of speech after 
which the recognition results are returned. Ignored 
for recos of modes other than automatic. If 
unspecified, defaults to platform internal value. 

» reject: Optional. The recognition rejection 

threshold, below which the platform will throw the 
^ no reco r ev^nt . If not specified,, the speech 
platform will use a default value. Confidence scores 
range between 0 and 100 (integer). Reject values lie 
in between. 

• server: Optional. URI of speech platform (for use 
when the tag interpreter and recognition platform 
are not co-located) . An example value might be 
server^protoco I : / /your spe&chplat form . An application 
writer is also 1 able to provide speech platform 
specific settings by adding a guerystring to the URI 
string, eg 

protocol : / /yourspe,Qchplatform?bargeinEnorgyThreshold 
-0.5. 

• langID: Optional. String indicating which language 
speech engine should use. The string format follows 
the xml:]ang definition. For example, langID="en-us" 
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denotes US English. This attribute is cnly effective 
when the langID is not specified in the grammar 
element (see 2.1-1). 
* mode: Optional. String specifying the recognition 
mode to be followed. If unspecified, defaults to 
"automatic" mode. 

2.2.2 Properties 

The foilowiny properties contain "the results returned by the 
recognition process (these are supported by uplevel 
browsers) . 

. recoResulb Read-only. The results of recognition, held 
in an XML DOM node object containing semantic markup 
language ( SML ) , as described in 2.1.2, In case of no 
recognition, the property returns null, 
text Read/Write. A string holding the text oC the 
words recognized (i.e., a shorthand (Tor contents of the 
text attribute of the highest level element in die SEL 
recognition return in recoResult in a read mode. In 
wr ite mode, a string can be assigned, which will then 
be parsed as if the string corresponds to the 
recognition result. The write mode allows extension of 
the markup tags and processing thereof to other 
components or applications on the client device. The 
string can be obtained from the "smex" message object. 

status: Read-only. Status code returned by the 
recognition platform. Possible values are 0 for 
successful recognition, or the failure values -1 to -4 
(as defined in the exceptions possible on the Start 
method (section 2.3.1) and Activate method (section 
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2.3.4) ), and statuses -11 to -15 set on the reception 
of recognizer evenly (see 2.4) . 

2*3 Ohjeat methods 

Reco activation and grammar activation may be controlled 
using the following methods in the Reco' s DOM object. With 
these methods, uplevei browsers can start and stop Reco 
objects, cancel recognitions in progress, and activate and 
deactivate individual grammar top-level rules {uplevei 
browsers only) . 

2.3.1 Start 

The Start method starts the 
active grammars all the top 
context which have nor been 

Syntax : 

Obj ect . Start ( ) 
Return value : 

None . 
Exception : 

The method sets a non-zaro status code and fires 
an onNoReco event when fails. Possible failures 
include no grammar (reco status -1) , failure to 
load a grammar, which could be a variety of 
reasons like failure to compile grammar, non- 
existent ORI (reco status = -2), or speech 
platform errors (reco status = -3), 

2.3.2 Stop 

The Stop method is a call to end rhe recognition process. 
The Reco object stops recording audio, and the recognizer 



recognition process, using as 
-level rules for the recognition 
explicitly deactivated . 
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re L urns recocnition results on the audio received up to trie 
point where recording was stopped. All the recognition 
resources used by Reco are released, and its grammars 
deactivated. (Note that this mezhod need not be u^ed 
explicitly for typical, recognitions in automatic mode, since 
the recognizer itself will stop the reco object or. endpoint 
detection after recognizing a complete sentence.) If the 
Pveco has noL been started, the call has no effect. 



Syntax : 

Object. Stop ( ) 
Return value : 

None , 
Exception : 

None , 

2.3.3 Cancel 

The Cancel method stops the audio feed tc the recognitor, 
deactivates the grammar and releases the recognizer and 
discards any recognition results. The browser will disregard 
a recognition result for canceleo recognition. If the 
recognizer has not been started, the call has no effect. 



Syntax : 

Object .Cancel ( ) 
Return valna : 

None. 
Exception : 

None . 
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2.3.4 Activate 

The Activate method activates a top-level rule in the 
context free grammar (CFG) . Activation must be called before 
■recognition begins, since it will have no effect during a 
'Started' recognition process. Note that all the grammar 
top-level rules for the recognition context which have not 
been explicitly deactivatea are already treated as active. 

Syntax : 

Object. .Activate (strName) ; 
Parameters : 

o strName: Required. Pule name to be activated. 
Roturn value: 

None . 
Exception : 

None . 

2.3.5 Deactivate 

The meLhod deactivates a top-level rule in the grammar. 
If the rule does not exist, the method has no effect. 

Syntax : 

Obj ect. Deactivate (strName) ; 
Parameters : 

o strName: Required. Rule name to be deactivated. An 

empty string deactivates all rules. 
Return value 

None. 
Exception 

None . 
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2 . 4 _Reco events 

The Reco DOM object supports the (following events, whose 
handlers may be specified as attributes of the reco element. 

2.4.1 onReco : 

This event, gets tired when the recognizer has a 
recognibion result available for the browser. For recos 
in automatic mode, this event stops the recognition 
process automatically and clears resources (see 2.3.2). 
OnReco is typically used for programmatic analysis of 
the recognition result and processing of the result 
into the page. 

Syntax : 



Inline HTM1 


<Reco onReco =' f handler" > 


Event property 


Ob] oct . onReco = handler} 
Ob j ect . onReco = 
GetRef ("handler") ; 



Event Object Info: 



Bubbles 


No 


To invoke 


User says something 


Defaih r_ 
QCtion 


Return recognition result object 



Event Properties : 

Although the event handler does not receive properties 
directly, the handler can query the event object for 
data (see the use of the event object in the example 
below) . 
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Example 

The following XHTML fragment uses onReco to call a 
script to parse rhe recognition outcome and assign the 
values to the proper fields. 

<input name^"txtBoxOrigin" tvpe="text" /> 
<input name="txtBoxDest* type-"text" /> 
<reco or_Reco^"procGSsCityRecogni tion ( ) "/> 

<grarnmar src= "/grammars / cit ies . xml" / > 

</reco> 

<script><! [CDATA; 

function processCft yRecognition () { 

smlRGSult = event . srcElGmsnt . recoResu 1 1 ; 

origNode = 

sml Result - selectSingleNodG (^//origin _city") ; 

if ( origNode ! - null) txtBoxOr tgin . value = 

ori gNode . text; 

destNode = 

smlResult . selectSingleWode { V/dest_c.ity" ) ; 

if (destNode null) txtEoxDest . valup -= 

destNode . text ; 

} 

] ] ></script> 
2.4.2 onSilence: 

onSilence handles the event of no speech detected by the 
recognition platform before the duration of tine specified 
In the initialTimeout attribute on the Reco (see 2.2.1). 
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This event cancels the recognition process automatically tor 
the automatic recognition mode. 



Syntax : 



Inline HTML 


<reco onSilence="handler" .„> 


Event property (in 
hCMAScript ) 


Object . onSilence = handler 
Ob j ect . onSilence = 
GetRef {"handler") ; 



Event Object rnfo : 



Bubbles 


No 


To invoke 


Recognizer did not detect speech within 
the period specified in the 
initial'! imeout attribute. 


Default 
action 


Set status — -1". 



Event Properties : 

Although the event handler does not receive properties 
directly, :he handler can query the evenL object for 
data . 



2.4.3 onTimeo-u 'c 

or.riir.eout handles two types of event which typically reflect 
errors firom the speech platform. 

• It handles the event thrown by the tags interpreter 
which signals that the period specified in the Tiaxtime 
attribute [see 2.2.1) expired before recognition was 
completed. This event will typically reflect problems 
that could occur in a distributed architecture. 
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• It also handles (ii) the event thrown by the speech 
recognition platform when recognition has begun bur 
prccessing has stopped without a recognition within the 
period specified by babbleTimeout (see 2.2.1) . 

This event cancels the recognition process automatically. 



Syntax : 



Inline HTML 


<r^co o nT ime out-="i2 an dler" ..,> 


Event property [in 
ECMAScript) 


Object . onrimeDut — handler 
Obj ect . on Tine Out = 
GetRef {"handler") ? 



Bvant Ob j act Iniro : 



Bubbles 


No 


To invoke 


Thrown by the browser when the period 
sot by tho maxtime attribute expires 
before recognition is stepped. 


Default 
action 


Set reco status to -12. 



Event Properties : 

Although the event handler does not receive properties 
directly, the handler can query the event object for 
data . 



2.4.4 onNoReco : 

onNcReco is a handler for the event thrown by the speech 
recognition platform when it is unable to return valid 
recognition results. The different cases in which this may 
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happen are distinguished by status code. The event stops the 
recognition process automatically. 



Syntax : 



Inline HTML 


<Reco onNoReco -"handler" > 


Event property 


Object . onNoReco = handler? 
Object . onNoReco 
GetRef ("^ndler" ) ; 



Event Object Info: 



Bubbles 


NO 


To invoke 


Recognizer detects sound but is unable 
to interpret the utterance. 


Default 
action 


Set status property and return null 
recognition result. Status codes are set 
as follows : 

status -13: sound was detected but no 
speech was able to be interpreted; 
status -14: some speech was detected and 
interpreted but rejected with 
insufficient confidence (for threshold 
setting, see the reject attribute in 
2.2.1) . 

status -15: speech was detected and 
interpreted, but a complete recognition 
was unable to be returned between the 
detection o: speech and the duration 
specified in the babbleTimeout attribute 
(see 2.2. 1) , 
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Klvent Properties : 

Although the event handler does not receive properties 
directly, the handler can query the event object for 
data . 

3 Prompt 

The prompt element is used to specify system output. Its 
content may he one or more ot the following: 

• inline or referenced text, which may be marked up with 
prosodic or other speech output information; 

• variable values retrieved ar render time from the 
containing document; 

• links to audio files. 

Prompt eleinenrs may be interpreted declarative ly by 
downlevel browsers (or activated by SMIL commands) , or by 
object methods on uplevel browsers. 

3.1 Proirpt content 

The prompt element contains the resources for system output, 
either as text or references to audio files, or both. 

Simple prompts need specify only the text required fox- 
output, eg: 

<prompt id="Welcorae"> 

Thank you for calling ACME weather report. 
</ prompt > 
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This simple text may also contain further markup of any of 
the kinds described below. 

3.1.1 -Speech Synthesis markup 

Any format of speech synthesis markup language can be u^ed 
inside the prompt element. (This format may be specified in 
the ^tts' attribute described in 3.2-1.) The following 
example shows text with an instruction to emphasize certain 
words within it: 

<prompt id="giveBalance""> 

You have <emph> five dollars </emph> .left in your 

account . 

</p rompt> 

3.1.2 Dynamic content 

The actual content of the prompt may need to be computed on 
the client just before the prompt is output. In order to 
confirm a particular value, for example, the value needs to 
be dereferenced in a variable. The value element may be 
used for this purpose. 

Value Element 

value: Optional. Retrieves the values of an element in the 
document . 

Attributes : 

• tatgetEl^ment; Optional. Either href or targe tElement 
must be specified. The id of the element containing the 
value to be retrieved. 

• targe tAttr ibu tG : Optional. The attribute of the element 
from which the value will be retrieved- 
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* href: Optional. The URI of an audio segment, href will 
override tar get Element i i: both ore prftRfint. 

The t argetElement attribute is used to reference an elemenr 
within the containing document. The content of the element 
whose id is specified by tar get Element is inserted into the 
text to be synthesized. If the desired contenL is held in 
an attribute of the element, the targetAt tribute attribute 
may be used to specify the necessary attribute on the 
tcrgetElement . This is useful for dereferencing the values 
in HTML form controls, for examp'o. In [;he following 
illustration, the "'value" attributes of the v> txtBoxOrigin" 
and ^txtBoxDest" elements are inserted into the tex; before 
the prompt is output 

<prompt id =" Conf;Lrrti"> 

Do you- want to travel from 
< value targetElement-="txtBoxOrigin" 
t a rg nt Aft r i hut e =" va 1 u e " /> 
tc 

< value targetEle-Tient= f/ txi:BcxDest" 
tar get Attribute^" value" /> 
7" 

</prompt> 
3.1,3 Audio files 

The value element may also be used to refer to a pre- 
recorded audio file for playing instead of, or within, a 
synthesized prompt. The following example plays a beep at 
the end of Che prompt: 

<prompt> 
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After the beep, please record your message. 
< value hrGf="/wav/bcep.wav ;/ /> 
</prompt> 

3.1.4 Referenced prompts 

Instead of specifying content inline, the sic attribute may 
be used with an empty element to reference external concent 
via URI r as in: 

<prompt id =// Welcome" src-'VACMEWeatherPromptsfWelcome" 

/> 

The target of the src attribute can hold any or all of the 
abo^e content specified for inline prompts. 

3.2 Attributes and properties 

The prompt element holds the following attributes (downlevel 
browsers) and properties (downlevel and uplevel browsers) . 

3.2.1 Attributes 

* tts : Optional. The markup language type for text-to- 
speech synthesis- Default is "SAPI 5'\ 

* src: Optional if an inline prompt is specified. The 
rjRl cf a referenced prompt (see 3.1.4). 

* bargeln : Optional. Integer. The period of time in 
milliseconds rrom start of prompt to when playback 
can be interrupted by the human listener. Default is 
infinite, i_e. r nu bargein is allowed. Bar gein=0 
allows immediace bargein. This applies to whichever 
kind of barge-in is supported by platform. Either 
keyword or energy-based bargein times can be 
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configured in this way, depending on which is 
enabled at. the time the reco is smarted. 
• pre-ffetch: Optional. A Boolean flag indicating 

whether the prompt should be immediately synthesized 
and cached dt browser when the page is loaded. 
Default is false. 

3.2.2 Propertias 

Uplevel browsers support the following properties in the 
prompt's DOM object. 

bookmark: Read-only. A string object recording the text 
of tha las~ synthesis bookmark encountered, 
status: Read-only. Status code returned by the speech 
plat f orrr. . 



innerbGxt: Read-only. This property will provide the 
text transcription of the prompt, which would be sent 
to the synthesizer. Fur instance, if a prompt comprises 
playing an audio wave file, this property provides a 
cext version of that prompt {often stored wxth the 
audio wave file), which can then be displayed, or 
otherwise used, for example, by providing the text 
version of the prompt to a component or application 
running on the client device. The innertext property 
can also be used to provide text versions of prompts 
containing dynamic content. 

3 . 3 Prompt methods 

Prompt playing may be controlled using the following methods 
in the prompt's DOM object. In this way, uplevel browsers 
can start and stop prompt objects, pause and resume prompts 
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in progress, and change the speed and volume of the 
synthesized speech. 

3.3.1 Start 

Start playback of the prompt. Unless an argument is 
given, the method plays the contents of the object. 
Only a single prompt object is considered 'started' at 
a given time, so if Start is called in succession, all 
playbacks are played in sequence. 

Syntax : 

Obj ect . Start ( [srrText ] J ; 
Parameters : 

o s tr Tex t z the text to be sent to . the synthesizer. 

If present r this argument overrides the contents 

of the object. 
Return value: 

None . 
Exception : 

Set status = -1 and fires an onComplete event if 
the audio buffer is already released by the server. 

3.3.2 Pause 

Pause playback without flushing the audio buffer. This 
method has no effect if playback is paused or stopped. 

Syntax : 

Object . Pause ( ) ; 
Return value: 

None . 
Exception : 

None . 
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3.3.3 Resume 

Resume playback 
method has no effect 

Syntax : 

Ob jec L . Res urae ( ); 
Return value : 

None . 
Exception : 

Throws an exception when resume fcils. 

3.3.4 Stop 

Step playback, if not already, and "lush the audio 
buffer. II the playback has already been stopped, the method 
simply flushes the audio buffer. 

Syntax : 

Ob j ecu. Stop ( ) ; 
Return value : 

None. 
Exception: 

None . 

3.3.^ Change ( 

Change speed and/or volume of playback. Change may be 
called during playback. 

Syntax : 

Obj ect . Change ( speed, volume ) ; 
Parameters : 

o speed: Required. The factor :o change, Spood-2 . 0 
means double the current rate, speed=0,5 means 



without flushing the audio buffer- This 
if playback has not been paused. 
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halve the current rate, speed=0 means to restore 

the default value, 
c volume: Required. The factor to change. Volume=2 . 0 

means double the current volume, volume -0.5 means 

halve the current volume r volume -0 means to 

restore the default value. 
Return value: 

None . 
Exception : 

None . 



3,3. £ Prompt control example 

The following example shows how control of the- prompt using 
the methods above might be authored for a platform which 
docs not support a keyword barge- in mechanism. 

<html> 

<ti-le>Prompt controK /t it lc> 

<head> 
<s cript> 
< J — 

function checkKWBargein ( ) { 

news . change (1 . 0, 0.5); // curri down the volume 
while verifying 

rf {keyword, text == 71 " ) \ // result is below 
eshol d 



thre: 



news-change (1- 0, 2.0); // restore the volume 
keyword. Star- () ; // restart the recognition 



> else { 

news . Stop {) ; // keyword deLected! Stop the 

prompt 

// Do whatever that is necessary 

} 

} 

;/ 

</script> 
<scrrpt fcr "window 11 event= TT onload ,, > 

< I — 
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news. Start () ; keyword. Start () ; 

// 

</script> 
</head> 
<body> 

<pronrpt id= T1 news rT bargei r="0 n > 

Stocks turned in another lackluster performance Wednesday 

as investors received little incentive to make any big moves 

ahead of next week's Federal Reserve meeting. The tech-heavy 

Nasdaq Composite £ndex dropped 42.51 points to close at 

215 6.25, The Dow Jones Industrial Average tell 17.05 points 

to 108 66.4 6 after an early-af bsrnoon rally failed. 

- < ! — 
</piompt> 
< reoo id= n keyword r ' 
ig ject="70" 

onReco="checkKWBarggin() " > 

< grammar src- http : //denali/news barge in gra?nmar . xml /> 

</reco> 
</body> 
</ht.Td> 

3 . 4 Prompt events 

The prompt DOM objecT. supports the following sveuLs, whose 
handlers rr.ay be specified is attribute.? of the prompt 
element . 

3.4.1 onBookmark 

Fires when a synthesis bookmark is encountered* The 
event does not pause the playback. 



Syntax : 



Inline HTML 


<prompt onBoakinark=" handler" 
...> 


Event property 


Object . onBookrnark = handler 
Object . onBooVrmark - 
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CetRef ("handler") ; 



£vent Object Info: 



Bubbles 


No 


To invoke 


A bookmark in the rendered string is 
encountered 


Default 
ac Lion 


Returns the bookmark string 



Event Prop^rhiGS : 

Although Lhe event handler does not receive properties 
directly, -he handler can query the event object for 
data. 

3.4.2 onBargein : 

Fires when a user's barge-in event is detected. ( No L e 
that determining what constitutes a barge-in event, eg 
energy detection or keyword recognition, is up to the 
platform.) A specification cf this event handler does 
nor automatically turn the barge -in on. 



Syntax : 



Inline HTML 


< p r oiTTp t onE argein = "handler'' 
.„> 


Event property 


Object . onBargein - handler 
Object . onBargein = 
■Set Kef ("handler") ; 



Event Object Info: 



Bubbles 


Wo 


To invoke 


A bargain event is encountered 
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Default 


None 


action 





Event Properties : 

Although the event handler does not receive properties 
directly, the handler can query the event object fur 
data . 

3.4,3 on Complete : 

Fires when the prompt playback reaches the end or 
exceptions (as defined above) are encountered. 



Syntax : 



Inline HTML 


<prompt onComplete =" handler" 
._> 


Event, property 


Object. onComplete = hcindl^r 
Object. onComplete = 
CetRef ("handler") ; 



Event Ob j ect Info : 



Bubbles 


No 


Jo invoke 


A prompt playback completes 


Default 
action 


Set status = 0 if playback completes 
normally, otherwise set status 
specified above. 



Event Properties: 

Although the event handler does not receive properties 
directly, the handier car. query the event object for 
data . 
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3.4.4 U*s±ng- bookmarks and. events 

The "following example shows how bookmark events can be used 
to determine the semantics of a user response - either a 
correction to a departure city or the provision of a 
destination city - in terms of: when bargein happened during 
the prompt output. The onBargein handier calls a script 
which sets a global Vxiark/ variable to the last bookmark 
encountered in the prompt, and the value of this hnark' is 
used in the reco's postprocessing function ( A heard r ) to seL 
the correct value. 



<script><! [CIjATA[ 
var mark; 

function interrupt ( ) { 

mark — event - srcElemont . bookmark; 

} 

function ProcessCityConf irm ( ) ( 

confirm. stop ( ) ; // flush the audio butter 
if (mark == xx mark origin city") 
txtBoxOrigin . value = 
event . srcElement . text ; 

else 

txtBoxDe st . value = 
event , srcElement. . text; 
} 

] ] ></script> 
<body> 

<input name^'txtScxOrigii]" value^^Seattle'' 
type="tex J L / V> 

<input narne=' / txtBoxDest' A type= /f text" /> 

<prompt id-"confirm f/ onBargein^"interrupt ( ) /f 

bargein="0"> 

From <b co km ark nark= /f mark_origin _city" /> 

<value targe tElement=" or gin" 
target At tribute-lvalue" />, 

please say <bookmark mar k="ma rk_des cecity" /> the 

destination city you want to travel to. 
</ prompt > 

<reco on.Reco="ProcessCityC:onf irm ( ) " > 

<grammar src="/grm/ 1033/citie.s . xml" /> 
</reco> 
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</body> 
4 DTMF 

Creates a DTMF recognition object. The object can be 
instantiated using inline markup language syntax or in 
scripting- When activated, DTIM K 1 can cause prompL object to 
lire a barge- in event. It should be noted the tags and 
eventing discussed below with respect to DTMF recognition 
and call control discussed in Section a generally pertain to 
interaction between the voice browser 21.6 and media server 
214 . 

4 . 1 Content 

• dtm^grainmar : for inline grammar. 

bind: assign DTMF conversion result to proper field. 
Attributes : 

• targetE lament: Required* The e lenient to which a partial 
recognition result will be assigned to (cf . same as in 
W3C SMIL 2.0 ) . 

• 1 targe tAt tribute : the attribute of the targe: element to 

which the recognition resul: will be assigned to (cf. 
same as in SMIL 2.0 ). Default is Value". 
» test: condition for the assignment. Default is true. 

Example 1: map keys to text 

<input typ^="text" name="city'V> 
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<DTMF id="ciry__choice" tlmeout^'^OOO^ numDigits="l"> 
<dtmf gramma r> 

<key ^lue= // l' / >Sea Ltle</key> 

<lcey value^"2">Boston</key> 
</dtrnf grammar > 

<bind tarqetElemenl-="c±ty" taru"etAttribute ^'value 

/> 

</DTMF> 

y choice" is activatGd, "Seattle" will be 
to the input field if the user presses l r 
if 2, nothing otherwise. 



Example 2: How DTMF can be used with multiple fields, 

<input type^tex;'' name= // area_code' r /> 
<mput rype="tex-" na ms= / 'phone__number / ' /> 
<DTMF id="areacode" numDigits="3" 
onReco=' / exten3ion.Activate [) "> 

<bind targetElemen"i:="a2:Ga_codG" /> 

< / DTMF> 

<DTMF id= // Gxtension" numDig it &=" 7 "> 

<bind targetElement="phcne_nunibcr' / /> 
</DTMfc> 

This example demonstrates how to allow users entering 
into multiple fields. 



Example 3: How to allow boLh speech and DTMF inputs and 
disable speech when user starts DTJMF . 



<input type="text // name= / 'credit_card_number' r /> 
<prompt onBookmark ="dtmf .Start: () ; speech . Start ( ) 
bargein="C"> 
Ples.se say <bookmark name=" star ting'' /> 
or enter your credit card number now 
</prompt> 

<DTMF id="dtir.f" escape-"*" length="16" 
interdig±tTimeout="2 000" 

onkeypress="speech. Stop () "> 

<bind targetElement=''credit_card number" /> 



When " c i c 

assigned 

"Boston" 
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</DTMF> 

<rpco id="speech" > 

<grammar src =/ 7grm/ 1033/digit s .ami" /> 
<bind t.argeLElGmerit = // credit_card_number // /> 

</reco> 



4,2 Afcfcri2>i3t:e.s and p2rope2rti.es 

4.2.1 Attributes 

• dtmf grammar : Required. The URI of a DTMF grammar. 

4.2.2 Properties 
DTMFgrammar Read-Write. 

An XML DOM Node object representing DTMF to string 
conversion matrix (also called DTMF grammar) . The 
default grammar is 



<dtmf grammar > 

<key value="0">O</key> 
<key value="l">K/key> 

<key value="9">9</key> 
<key value="*">*</key> 
<key valuG = r/ # // >#</ksy> 
</ dtmf grammar > 

• flush 

Read-write, a Boolean flag indicating whether to 
automatically flush the DTMF buffer on the underlying 
telephony interface card before activation. Default is 
false to enable type-ahead. 



• g scape 

Read-Write, The escape key to end the DTMF reading 
seGsion. Escape key is one key. 
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numDiglts 

Read-Write. Number of key strokes to end the DTMF 
redding session. If both escape and length are 
specified, the DTMF session is ended when either 
condition is met. 

dtmfResult 

Read -only string, storing the DTMF keys user has 
entered. Escape is included in result if Typed. 

• text. 

Read-only string storing white space separated token 
string, where each token is concerted according to OTMF 
grammar . 

in.Lt.ia ITimeout 

Read-write. Timeout period for receiving the first DTMF 
keystnke, in m i 1 1 i. seconds . If unspecified, defaults to 
the telephony platform's internal setting. 

interdigitTiirieout 

Read-Write. Timeout period for adjacent DTMF keystokes, 
in milliseconds. If unspecified, defaults to the 
telephony platform's internal setting. 

4 . 3 Oh j&ab methods : 



4.3.1 Start 

Enable DTMF interruption and start a DTMF reading 
session . 
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Syntax : 

Object . Start ( ) ; 
Return value : 

None 
Exception : 

None 



4.3.2 Stop 

Disable DTMH 1 , The key strokes entered by :.he user, 
however, remain in the buffer. 

Syntax : 

Obj ect . Stop ( ) } 
Return value: 

None 
Exception : 

None 

4.3.3 Flush 

Flush the DTMF buffer. Flush can not be called during a 
CTMF session . . 

Syntax : i 

Object. Flush ( ); 
Return valuer 

None 
Exception : 

None 
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4, 4 Events 

4,4.1 onkeypress 

Fires when a DTMF key is press. Phis overrides the 
default event inherited from the HTML control. When 
user hits the escape key, the onRec event fires, not 
onKeypress . 



Syntax : 



Inline HTML 


<DTMF onkeypress="nandler" ...> 


Event property 


Or 1 j ect . onkeypres s — handler 
Object .onksypress = 
GetRef ("handler") ; 



Event Object Info: 



Bubbles 


No 


To invoke 


Press on the touch-tone telephone key 
pad 


Default 
action 


Returns the key being pressed 



Evont Properties : 

Although the event handler does not receive properties 
directly, the handler con query the event object for 
data . 



4.4.2 onReco 

Fire's when a DTMF session is ended. The event disables 
the current DTMF object automatically. 
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Syn tax : 



Inline HTML 


<DTMF onReco=" handler" ...> 


Event property 


Obj ect . onReco = handler 
Object. onReco = 
GetRef ["handler") ; 



Event Object Info : 



Rubbles 


No 


To invoke 


User presses the escape key or the 
number of key strokes meets specified 
value . 


Default 

action 


Returns the key being pressed 



Event Proportias: 

Although the event handler does nob receive properties 
directly r the handler can query the event object for 
data . 

4.4.3 onTimeoufc 

Fires when no phrase finish event is received before 
time out. The event halts the recognition process 
automatically . 



Syntax : 



Inline KTML 


<DTMF onTimeout=" handler" ...> 


Event property (in 
ECMA8cri.pt ) 


Obj ect . onTimeout = handler 
Object . nnTimeout = 
GetRef ; "hand ler rr ) ; 



Event Object Cnfo r 
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Bubbles 


No 


To invoke 


No DTMF key stroke is detected wichin 
the timeout specified. 


Default 
action 


None 



Event Properties: 

Although the event handler does no: receive properties 
directly, the handler can query the event abjech for 
data . 



5 CallControl Objoct 



Represents the telephone interface (call, terminal, and 
connection) of the Lelephone voice browser. This ooject is 
as native as window object in 2 GUI browser. As such, the 
lifetime o iT the telephone object is Lhe same as Lhe browser 
instance iLself. A voice browser for telephony instantiates 
the telephone object, one for each call. Users don't 
inoLantiate or dispose the object. 

At this point, only features related to firsc-party call 
controls are exposed through this object. 

5 . I P.rqpe:zrtie.s 
address 

Read-only. XML DOM node object. Implement a Ci on 
specif. ic- This is the address of the caller. For P3!N r 
may a combination of AN! and All. For VoIP, this is the 
caller's IP address- 
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♦ rincfsBef orei\nswGr 

Number of rings before answering an Incoming call. 
Default is infinite, meaning the developer must 
specifically use the Answer ! ) method below bo answer 
the phone call. When the call center uses ACQ to queue 
up the incoming phone calls, this number can be set to 
0 . 

5.2 Methods 

Note: all the methods here are synchronous. 

5.2.1 Transfer 

Transfers the call. For a blind transfer, the system 
may terminate the original call and free system 
resources once the transfer completes. 

Syntax : 

telephone . Transfer (strText ) ; 
Parameters : 

o strText: Required- The address of the intended 
receiver . 
Return value : 

None . 
Exception : 

Throws an exception when the call transfer fails, 
e.g., when end party is busy, no such number, fax or 
answering machine answers. 

5,2.2 E -ridge 

Third party transfer. After one call is transferred, 
the browser may release resources allocated for the 
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call. It is up to the application to recover thy 
session state when the transferred call returns using 
strUID. The underlying ieleohony platform may route the 
returning call to a different browser. The call can 
return only when the recipient terminates the call. 

Syntax : 

telephone- . Rr i dge (strText , strUID, [imaxTirte] ) ; 
ParametGrs : 

o strText: Required. The address of the intended 
receiver . 

o strUID: Required. The session ID uniquely 

identifying the current call. When the transferred 

call is routed back, the srtOID will appear in the 

address attribute . 
o imaxTimG ; Optional, Maximum duration in seconds of 

the transferred call. If unspecified, defaults to 

p 1 a t f orrn-i n ternal val ue 
Return value; 

None . 
E^ccep tiorx : 

None . 

i 

5.2.3 Answer 

Answers the phone call. 

Syntax: 

telephone . Answer [ ) ; 
He turn value : 
None . 
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Excep tion : 

Throws an exception when there is no connection. 
No onAnswer event will be fired in this case. 

5.2.4 Hangup 

Terminates the phone call- Has no effect if no call 
currently in progress. 

Syn tax : 

telephone . Hangup ( } ; 
Return value: 

None . 
Exception : 

None . 

5.2.5 Connect 

Starts a first-party outbound phone call. 

Syntax: 

telephone . Connect { strText r [ iTimeout ] ) ; 
Parameters : 

o strText : Required. The address of the intended 
receiver . 

o iTimeout: Optional. The time in milliseconds 
before abandoning the attempt. If unspecified, 
defaults to platform-internal value. 

RGturn value : 
None . 

Exception : 
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Throws an exception when the call cannot be 
completed, including encountering busy .signals or 
reaching a FAX or answering machine (Note: 
hardware may not support this feature) . 

5.2.6 Record 

Record user audio to file. 

Syn tax : 

telephone . Record [url, ends 11 en ce, [max Timeout] , 
[ init ial Time out] ) ; 
Parameters : 

a url: Required. The url of the recorded results. 

o end^ilence : Required. Tine in milliseconds Lo stop 

recording after silence is detected, 
o maxTxmeout; Optional. The maximum time in seconds 

for the recording. Default is plat form- specific, 
o iuiti^alTimeout : Optional. Maximum time (in 

milliseconds) of silence allowed at the beginning 
of a recording. 
Return value : 

None . 
Exception : 

Throws an exception when the recording can not be 
written to the url. 

5.3 Event Handlers 

App developers using telephone voice browser may implement 
the following event handlers. 
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5.3.1 onlncaniLng- ( ) 

CaHea when the voice browser receives an incoming 
phcne call. All developers can use this handler to read 
caller's dddxesy and invoke (customized features before 
answer.i ng the phone call , 



5.3,2 onAnawer ( } 

Called when the voice browser answer.? an incoming phono 
call. 



5.3,3 onHangup ( ) 

Called when user hongs up the phone. This even - , is NOT 
automatically fired when the program calls the Hangup 
or Transfer met nods. 



5 . 4 Example 

This example shows scripting wired to the call control 
events to manipulate the telephony session. 



<HTML> 
<KEAD> 

<TTTL£>Logon Page</T i TLE:> 

</HEAD> 

<SCRIPT> 
var focus; 

f an c t i on Run Sp e ec h ( } { 

if ( logon . user . value =- " " ) { 
f ocus-= "user" ; 

p_usd . Start (; ; g_login . Soar t ( ■ ; dtmf . St a r ;:. [ ) ; 

return ; 
} 

if {logon-pass . value ■== "") { 
f ocms="pin" ; 
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p_p in . Star l () ; g_login . Start () ; dtmf . Start () ; 

return; 
} 

p_thank . Start { ) ; logon , submit [ ) ; 

} 

f 11 n c 1. 1 o n 1 og 1 n_r e c o ' ) { 

res — event . srcElement * recoResulr ; 
pNode = res . selectSingleNode ( " //uid" ) ; 
if (pNode != null) 

logon . user . value = pNode.xml; 
pNode = res . selec LSingleNode ( 11 //password" ) ; 
If (pNode != null) 

logon.pass.value = pNode.xml; 

} 

tunction dtmf reco() { 

res event . s rcElement . dtmf Result ; 
if [ Cocus == "user") 

logon.naer.yalne = res; 

else 

logon . pin .value = res; 

} 

</3CRIPT> 

<SCRIPT for ^"callControl" event = TT onInooi(iing- 11 > 

< I — ■ 

// read address r prepare customized stuff if any 
ca.l LC one rol . Answer ( ) ; 

// 

</SCRIPT> 

< SCRIPT f or-" callControl " even L = ,r onOff hook' 1 > 

< ! — 

p_main . Start () ; g_lo gin . Star L () ; dtmf . Start {) ; 
focu5^"user r ' ; 

/ / 

</SCRIPT> 

<SCRIPT for- n window" event="onload"> 
<! — 

if (logon . user . value !- "") { 
p retry. Sr. art ( ) ; 
logon . user * value = T11T ; 
logon, pass . value = " ,T ; 
checkFields ( ) ; 

} 

// 

</SCRI?T> 
<BODY> 

<reco id="g_log"in ,r 

onReco : ="login_reco () ; runSpescliO " 
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rirnecu--"500G 1T 

onTiir.eout="p_mi^<5, start () ; Hun Speech ( ) " > 

^grammar 

src= http : //kokanegl/etradedemo/speachQiily/login.mI /> 

</ reco > 
<dtmf id= Tr dtmf " 
escape-"!" 

onkeypress = T 'g_login . Stop {) ; 11 

on Re c-o= "dtmf_reco ( ) ; RunSpeach. ( ) T? 

incerdi git Timeout-' 7 5000" 

onrimeout = "dtmf . Flush ( ) ; p itiiss , Start () ;RunSp2ech () " /> 

<prompt i d" "pjaain n >PlGass say your user I D and pin 

number < /pr omp L > 
<proittpt "i- d="p__uid" >f?lease just <?ay your user I D< /prompt > 
<prompt 1 d= n p_pin">Please just say your pin number</prompL> 
<promp~ id= Tl p miss" > Sorry , I missed. tha.t< /prompt > 
<promp- i d = r, p_thank">Thank you . Please wait while X verify 

your ±dentity</prcrrrpt> 
<prorrpt id=' T p_retry ">Sorry r your user X D and pin number do 

not ma ten < / pr orr.pt > 

<H2 >Login< /H2 > 
<f o rrt 1 d- " log-on " > 

uxd : <1npi:r. nanie="user" type ■=" text" 
onChange^- "runSpeech () " /> 

PIN: < input name="pass" type = "password" 
onCriange = Tr RunSpeech () " /> 

<y'::.)rm> 
</B0DY> 
</'HTML> 



6 Controlling dialog- flow 



6.1 TJsLzxg HTML and script to Implement d±alu<j if low 

This example shows how to imptem^no a simple dialog flow 
which seeks values for input boxes and oiTfers context- 
sensitive help for the input. It usgs the title attrihulo on 
the HTML input, mechanisms (used in a visual browser as a 
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"tooltip" mechanism) to help form the content of the help 
prompt . 



<htrnl> 

<tit lG>Context Sensitive Help</~ itle> 
<head> 

< scrip I > vsr focu 5 ; 

function RunSpeech f > { 

if (trade. stock. value — "") { 
focus='"tradG . stock" ; 
p_stock.$tart{) ; 
return ; 

) 

if ( trade . op .value —— J { 
focus=" trade . op" ; 
p_op . 3 Uart () ; 
return ; 

} 

//.. repeat above for all fields 
trade . submit () ; 

) 

function handle () { 

res = event . sr c2 leman t . recoResul t ; 
if (res -text "help 11 > { 

text = " Please just say"; 

text +== document . all [focus] . title; 

p help . Start (text) ; 
} else { 

// proceed with value assignmenbs 
} 

} 

</5cript> 
</head> 
<body> 

< prompt id-"p_help TT cnCozrtplet e=' F checks! leds () " /> 

<prompt id= FT p__stock F1 onComplete= ,T g_stock.Sl;aT:t () ">Please say 

the =itoc]c nsme</prompt> 

< prompt id="p_op T1 onComp lei g "gop . Start <) " >Do you want to 
buy or selK/proinp L> 

<prompt id= M p_quantity" onComplete -"g_quantity . Start ( ) ">Hcw 
many 3hares?</prnmp_> 

<prompt id="p_j>rice" cnComp let e-"g_price'. Start () " >What r s the 
pricQ</p r omp t > 
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<reco id-"g_stock ,r onReco= "handle {) ; checkFields ( ) " > 

<grammar src— " . /g stoek.xml" /> 

</ reco > 

<reco id="g_op" onReco="handle {) ; checkFields () " /> 
<grammar" 3rc=" . /g__op, xml 1 ' /> 

</ reco > 

<reco id- TT g_qiiantitY' T onReco =" handle () ; checkFields ( ) " /> 
<grarrimar src- " . /g quant . xml " /> 

</ reco > 

<reco id= Tl g__price" onReco= M handle ( ) ; checkFields () " /> 
< grammar si c= " . / g_qudnt . xml Tr /> 

</ reco > 



<±orm id="trade"> 

<input name=" stock" title ="stock name" /> 
<select name- "op" title= T, buy or seLl"> 

<option v r alue="buy" /> 

<option value="seU r ' /> 

</sel Gct> 

<Input name^" quantity" t it le="nxunber o£ shares" /> 
<input name="price' F title= ,f prica" /> 

</form> 
</body> 
</hL:ml> 

6.2 Using- SMIL 

The following example shows activation of prompt and reco 
elements using SMIL mechanisms. 



<html xmlns : t-"urn: s enemas -micro soft-corn : time" 

xmlns : sp="urn : schemas-microsof t-com: speech"> 

<head> 
<style> 

.time { behavior: url (#def aul c#t ime2 ) ; } 
</s cyle> 
</head> 
<body> 

<input name-" txtBoxOrigin" type=" text n /> 
<input name^ r, txtBoxDost " type= Tf text" /> 
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<sp: prompt class=" t Ime" r : begin=" 0 " > 

Please say the oiigin and destination cities 
< / sp : prompt > 

<t:par t : begin= IT t ime . end" t : repeatCount=" indefinitely " 
- <sp : reco clas s= " time" > 

<grammar sre= " . /city . xml" /> 
<bind targecElerrent=="txtBoxOrigin rr 

value = ,T //origin_city T ' /> 
<bind targeLElement="txt:DoxDest " 
t-.est="/s3il/dest_city [(^contidence $gt$ 40]" 
value="//desr_city" /> 
</sp: reco 
</t :par> 

</body> 
</html> 



7 SMEX (Message) element/ object 



3MEX, short for Simple Messaging exchange /Extension, is an 
object that communicates with an external component or 
application on the platform of the client device. It can be 
embedded into an XML or similar markup based document as an 
element with tha tag name <smex>. Exemplary usages of ~hG 
messaging object may include logging and telephony controls. 
The object represents the extensibility of markup based 
recognition and prompting as it allows new fundtionality be 
□deed through messaging. 

Upon instantiation; the object Is directed to establish an 
asynchronous message exchange channel with a platform 
component or application through its configuration 
parameters or attribute specifications. It has a string 
property whose content is sent to the platform component or 
application whenever the property is the recipient of an 



(142) 03-131772 (P2003-131772A) 



assignment operation (i.e., lvalue). Similarly, it also has 
a property ol XML DOM Nude type that holds the message 
received from the platform component or application. The 
message object sends an event whenever it receives a 
platform message. Since its basic operations are 
asynchronous, the object also has a built-in clock for the 
application developers to manipulate timeout settings. 

The message or smex object is agnostic to the means of 
communications. However, in one embodiment , the smex object 
has the same life span as ordinary XML or markup elements, 
namely,, the smex object will be destroyed when its hosting 
document is unloaded. While in many cases, the smex object 
can perform automatic clean-up and release communication 
resources when it is unloaded, there might bo use cases 
(e.g., call controls) in which a persistent: communication 
link is desirable across markup pages. For those cases, the 
architecture places the responsibility of relinquishing the 
allocated resources (e.g. close the socket) on the 
application developers . 

The smex object is neutral on the format (schema) of 
rries sages. In some embodiments, it may desirable to require 
implementers to support a few rudimentary schemes, with 
strong preferences to existing standard messages formats 
(e.g. as used in SIP or CCXML) . In essence, the architecture 
allows both one platform and application developers to take 
-he fall advantage of the standardized extensibility of XML 
or similar markup to introduce other features without losing 
interoperability in the mean time, 

Example 1: use of smex as a logging object 
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<smcx id="logServer"> 

<param name="d: server" xmlns : d="urn : Microsoft- com/COM"> 
<d : protocol >nCOM</d : protocol > 

<d: clsid>20 9309 3 02 93 02 02 9320 94 20 93 4 320 9 8</d: clsid> 
<d: iid>0903 8 593D4 903 4 98 530 98h309094 8 03</d: iid> 
</parain> 
</smex> 

<listen .._> 

...// other direcLives binding reco resulLs to input 
fields 

<bind targetEieiiient="logSer-\rer" target At tribute="sent " 
value-"* [Slog $ge$ 3]"/> 

</listen> 

This example demonstrates how a .logging mechanism can be 
achieved using a COM object with iLs class id and interface 
id. The speech developers attach an attribute "log" 
indicating the level of interests i'or logging to the 
relevant SML nodes. In one example above ^ the app developer 
chooses to log all nodes with log value greater or equal to 
3 by using a single bind directive. The example works in 
both downlevei and uplevel browsers. 

The example also intends to demonstrate it is possible for a 
page to contain multiple smex objecLs communicated with the 
same platform component as long as there won' t be confusion 
on which smex object is responsible for delivering the 
platform messages back to the recognition document. The 
above example implies a component can implement multiple 
interfaces, each of which has its own smex or message 
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conduit. The same argument: applies to TCP servers listening 
to multiple ports. 

Example 2: Reading the addresses for an incoming call: 

< input type="text" id="remote r V> 
<input type="text" id~" transf er'7> 
<input type="text" id="iocaI"/> 
<input type=" hidden" id="session_id ,r / > 

<smex id=" telephone" sent^"start__list5ning y/ > 

<param namc=" server 'e>http: //tcl-svr/whatever</parain> 
<binc turgetElernsrvt ="sGSsion_id" valiie-'7/sid"/> 
<bind t arget El ement="r emote" value="//remote_addr'V> 
<bind targetElernent="transfer' / 

value='V /trans fer_addr"/> 

<bind targetE lsmerit=''local'' value="//local_addr"/> 

</sir.ex> 

This example shows how the bind directives can be used to 
process the received message. The example assumes a message 
fox incoming call to have sub-elements remote _addr r 
transfer_addr, and local_addr whose contents represent the, 
remote, transfer, and local addresses of the incoming call, 
respectively . 

In Lhis example, a connectionless protocol based on HTTP is 
used to communicate with the telephony server. The telephony 
server here is designed to communicate with more than ons 
browser instances, and hence each client must identi Fy 
itself with a unique id assigned by the server when the app 
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starts. This is achieved in this example by sending the 
server a x 'start__listening" message. In this example, the 
session id is stored in a hidden field that can be sent back 
to the Web server and passed on to the next page of the 
application, although other techniques (e.g. client side 
cookie) can also be used to manage session state. As is in 
the case lor oeco, not every bind directives will be 
executed for every platform message. The above example does 
not imply Lhe unique id will only be received when there is 
an incoming phone call. 

7 . 1 PzrapBirtl &s 

The smex object can have the following properties r with only 
the Read/Writ e properr.i es allowed to also serve as 
attributes for the initial value specification. 

• sent: Read/Write, a strung corresponding to Lhe message 
Lo be sent to the platform component. Whenever sent is 
used as a lvalue, its contents are dispatched. There is 
no effect when the property is used as a rvalue or when 
a null object is assigned to this property. 

9 received: Read Only, an XML DOM Node data indicating 
the received message. The message will be available as 
a rvalue until the next onReceive event is ready Lo 
send. * 

• timer: Read/Write, a number in millisecond indicating 
the time span before a timeout event will be triggered. 
The clock starts ticking when the property is assigned 
a positive value. The value can be changed when a count 
down is in progress. A zero or negative value stops one 
clock without triggering the cimeout event. The default 
is 0, meaning no timeout. 
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• atatus; Read Only, an integer indicating the recent 
status of the object. The possible values are 0, -1, 
and -2 r which means normal, timeout expired, and 
communication with the platform cannot be established 
or has been interrupted, respectively. Platform 
specific error messages should conveyed through the 
received property. For Lhe cases that the error message 
is successfully delivered, she status code is 0. 

7.2 Events 

The object has the following events: 

• onReceive: This event is sent when a plat iTorm message 
has arrived. If there are any directives declared by 
the bind elements, those directives will first be 
evaluated before ihe event is fired. Prior co the 
sending, the received property will be updated. 

• onError : This event is sent when the timeout expires, 
or a communication link error has been encountered. 
When the event is sent, the status property will be 
updated with a corresponding error code as described 
above . 

7.3 Child Elements 

When assuming an element form, sniex may have the following 
child elements ; 

• bind: same as in reco, except the directives are 
operated on the received message. 

• param: same as in reco, provides platform specific 
parameters for the smex object. Each param element may 
be named using a v name/' attribute, with the contents of 
the param element being the value of the parameter- In 
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one embodiment, the element should understand standard 
XML attributes for name spsiOQ and XML data type 
declaration . 



7.4 O th&jT C airmen ts 

One elegant way to extend SMTSX for the logging function 
would be 

<smex id="logServer" ...> ... </smex> 

<script> function logMessage (logClass, message) { 

logServer . sent = logClass + "|" ' ! ' message; 
} </script> 

This in effect extends the object with a (global) 
function whose behavior can be customized. In the 
above example, che logging function is programmed to 
insert a field separator " \ f/ between the id and the 
message . 

For those who do noL like global functions, they can 
use the "prototype" property of ECMAScript to attach 
the function as an object method. For example 

<siaex id="logServer" onload-^addFunction ( ) ... </smex> 
<script> 

function rny_logMessage (logClass , message) { 

logServer. sent = logClass + vv |" + message; 

} 

function add Function ( ) { 
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log Server . prototype . logMessage = 
my_ logMessage? 
} 

</ script> 

One can refer tc the function in a more objected- 
oriented manner: 

logServer. logMessage (R^CO_LOG_ERROR, "My 
message" ) ; 

It is noted that more work is required Trom the smex 
object implemenbers in order to make the extension work 
as in the above example, although all the necessary 
mechanisms are already well established standards. 



(149) 03-131772 (P2003-131772A) 



I. Brief Hescri pi. ' on of the Drawings 

FIG. 1 is a plan view of a first embodiment of a 
computing device operating environment. 

FIG. 2 is a block diagram of Lhe computing device of 
FIG. 1. 

FIG, 3 is a plan view of a telephone. 

FIG, 4 is a block diagram of a general purpose computer. 

FIG. 5 is a block diagram of an architecture for a 
clienr/server system. 

FIG. 6 is a display for obtaining credit card 
information. 

FIG. / is a page of mark-up language executable on a 
client . 

FIG. 8 is an GKampliry pnge of mark-up language 
executable on a client having a display and voice recognition 
capabilities . 

FIGS. 9A and 9B arc an exemplary page of mark-up 
language executable on a client with audible rendering only 
and system initiative. 

FIG. 10A and 10D are an exemplary page of mark-up 
language executable on a client with audible rendering only 
and mixed initiative. 

FIG. 11 is an exemplary scripc executable by a server 
side plug-in module. 

FIG. 12 is a pictorial illustration of a first 
operational mode of a recognition server. 

FIG. 13 is a pictorial illustration of a second 
operational mode of the recognition server, 

FIG. 14 is a pictorial. illustration of a third 
operational mode of the recognition server. 

FIGS. IbA and 15B are an exemplary page of declarative 
mark-up language executable on a client without scripting. 
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290-^ 



< htm I > 

<form id =* "g*t_ca rd_info w methou^^ost" actfcn = "http :/ / pa y men t.asp r > 

^select nam e- H c»rd Jvpe" onCljck = *talk( p „ca rd_ty p es) " > 

<option value ---a iiiex'> American- Exp ress< /option> 

< option value - "visa "> Visa < p tfon > 

< option vafue = "ms ,T > MasterCard < /opt ion > 
</5elect> 

<recu id= g_card_types" cn Reco =?"handl«[) " > 

* grammar sre - "./ 9 ra m #card_types" /> 
</ reco > 

<input type = "text* name^'card.num " w fdth = "30" 
o nClick = "ta lk(__card_n urn) " /> 
<reco id= " g^card _num " onReco hand le() " > 

< grammsr sre = " */g ra nn # dig it* " /> 
</reco ■> 

< input type = "text" nam e- * ex j>iry„d a te" ■■ - — ■ ■■■ 

on Click 'talk(g_e5cprry„Jate} n /> 
<reco id = "g_cxpiry .date " > 

< grammar sre = " ./gram #dates " /=* 
</reca > 

<input type- "submit 1 * va lu e = "Su bur it" onCiick 'v^rifyO" /> 

</form> ' ~~ 

<5Cnpt> <![CDATA[ 

function talk(gobj) { 
gabj.activateC); 

} 

function handle (} { 

if (get_caid_in£o. cai.d_nuxn != null) ( 

If (geL_cajd_inf o . card__typc . value — "antex") 
if ( get_ca rd_in fo . card^nuiu . length ! - 15) 
alert ("an-ex should have 15 digits"); 
> else 

if (gc?L_card info . card__num, length i — 16) 

alert ("visa and master should have I 6 digits"); 
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} 



function verify () [ 

var flag - Hindow.coufirmt "submit the credit card 
info?") ; 

if (flag) f 

yet_card_info.su.biT.it [ ] ; 

) 

> 



</SCript> 
</htm|> 



FIG. 8 



(157) 03-131772 (P2003-131772A) 



< body > 



Vform id-' 1 9 et_«rrf_1of0 " method --past 11 action = " http;/ /paym ent.asp " 
cnActivate =" welcom() -> 

< prompt id =" p_we!cc>me "> We now n«d yur credit crd </ prompt > 

< prompt id =" p mumble ^ I didn't understand you </ prompt > 

300 < prompt Id = "p_card_cypc " barg* in = - tr*. -> W h a t credit card would you 



302 



< prompt, iu - p > _ ■* " ^ ■ = — 

< prompt id =" p_e* P iry_d at * " b^rgein -"tr— Wh.t is the expiration 
date? </ prompt > 

-< prompt Id p .content "> „ 
I have your < value select = " card.typ* " /> < value $elect = card_n U .o 

with expiration date 

< value select = lb expiry_date " /> 

</ arompt > 

< prompt id - m p_carifirrO "> Is this correct? </prcmpt> 

< reco id"" §_card_typ c * " ^NoReco m u rti M*{ this, 3 ) "> 
onfteco = " _handtcCt!*'*» card^typ*) " /> 

< grammar sre » ■ ./ B ^m#c»rd JYP«' 

<Ve7oV=" o_card_num " onNoReco ^MmbteCthis, 1 J "> 
onRecc =" _handie{this, card_num) 
grammar sre = " ./ aram #dig Its " /> 
<Veco°id =" g_«plry_date " onNoReeo mumbl«(ttiit, 1) > 
onReco _handle(tiils f ±»iry date) " a udio - * rao " /> 

< grammar sre - ./0 ram #dates f> 

<V e r co C W=- confirm. tic." onR.co =• confirm ed(thi5) " /> 
c gammer src = " ,/ Q ram * yesnO /> 

< /reco > 

< select name =" card_type > 

< option value -"amax"> American Express </ option > 

< option value =" visa "> Visa option :> 

< option value =-«is"> MasterCard option ^ 
<:/ select > 

* input type="text" name =" card_nu.« * width 3» " /> 
^ input type -"text" rtam e = " expiry . date * /> 

< input type =" subm it * value =" Submit "/> 
</ form > 

< script > <= !(CDATA[ 

function welcome () I 
p welcome . active <) : 
repeat - 0; 
check Filled () ; 



1 



function ir.iunble (gob maxprompt$) { 

gob} . deactivate C) ; 

v mumble - active O i 
Checkril led{) ; 



TO FIG. SB 



FIG. 9 A 
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function _handleO { 
handleO; 
checkFinedO ; 

function checkFilledC) { 

if (caru_type. value ~= { 
p^card_type. activeO ; 
g_card_types .activateQ ; return ; 

if (card_nun?. value — { 

p.. cardan urn. active O ; g_card_nunr>acti vateO; return; 

if (expi ry_date .value — "") { 

p_expi ry_data.activeQ; g_expi ry^date .acti vateQ ; return; 

> 

p_con tent. act ivateO ; 
p_confirm. activate O ; 
cunfi rmaLion.acti vateO ] 



function ccmf irmedCg 0 ^) { 

if Cgobj . recogftes . text = "yes") 
get_card_info.subrrntO \ 



// user codes start here 
// 

function handle Q { 

if (get_card_info.card_type == "aniex 7 '^ { 

if (get_ca r d_i nfo . car d_nurru length ?= 15) i ( 
prompt .speak C'amex should Tiave 15 digits j; 
get_carrj_i:ifo.card_num = 



} 




> 

</scnpt> 



</body> 



FIG. 9B 
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<=body> , h 

<form id=," S ei_card_Jnfo" meticd^posL" action="http://payment.a5p 

onactivate= H wel«wne()'> . _ 

<prornpt id= '^welcome" > We now need your credit card</prornpt> 
<prompt id="p mumble w >I didn't understand you</prompt> 
<prompt id-"plcard_type w bargein-"Lrue*>What cred,t card would you 

u S e?</prampt> ^ ^ num "bargein-"true' 1 >Pi e as e 3 i 3yti 1 e nufnber</prornpt> 

<prompi. id=" P IexpirV_date" bar9ein="t. 1 ue">What is Che expiration 
date?</prompt> 

<prompt id- M p_e0ntenr> , > „ . , - 

I have your <vaiuc select = TH cartf„type' /><vd I ue select = card num /> 

with expiration date <vaiue sefect="^xpiry_date rT /=> 

</prampt> 

<prompt id-V_connrm">Is this correct?</prompt> fc i«.r«.i. ! v* 

<grarnmar -re -' ./gr^rn /get Payment' /> 
<bind targeL-"card_type" value=Yeard/typa" /> 
<bind target= w cart!_nurTi" value-Y^ard/number /> 
405 <bind tdrget="expiry_date" value- Vcard/expr.date" /> 

<^o°id="confirmation" onReco ^confirmed C«. is) onNoReco='<mumbl*<this}" 
<grammarsrc =",/^ram#yesno7> 

<s3ect : name= "cartf„iype"> 

<option value- "a mex n > American Express</option> 

< option value="visa'*>Visa<^option> 

^option v*iue= H ms">M*5terCard</option> 
< /select > 

<mpiJt type^ h text ,f n3rne= n ca.d_num n vvidth= 30 /> 
< input type *"text" nsme- r 'expiry_date" /> 
Onput type -"submit* value= n Submtt" /> 

</furrn> 

<?cript><HCDATAf 

function welcomeO i 
p^wel come . active O ; 
repeat ~ 0; 
chfickFiTledO: 



function mumbleCgobj) 
gob j . deatt i vateO I 
p_jnumbls.activeO; 
checkFi iledQ; 

function _handleO { 



FI6. 10 A 



} 



handleO: 
checkF-illedO; 



TO FIG. 10B 



T 
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Punction checknlled() I 
if (carcLtype. value == } i . 

p_card_tyi)e.icHveOr; do_fi eld. activate Q ; return, 

if CcardU mm. value { . 

p_card_num. active 0; do_Fi el d.activateO ; return; 

if Cexpi ry_date. value = { , _ . ^ . _ _ 

p_expiry„data. active Q; do . field. activate O: return, 

p_con t en t. activate O ; 
p_confi rm. activate C) ; 
confi rmation.activateO ; 

function confi nnedCgobj) { ■ 
if Csobj . recogRes.text — yes ; 

get_card_i nf o - suhnn" t (qensml Oj i 

} 

// user codes srart here 
function handle O { 

if Cfield =-- ge.t_card_info.card.num) { 

if (geC_cardLinfo,card,_nuni.lenqLh \-~ 15 J 4 M 
prompt. speak C'amex shoulu have 15 digits J ; 
get_card_in fo. card_num = ""; 

} 

} else , 
if (oetL_f ard_info . card_num. length != lbj p t rt 
prorept. speak ('Visa should have 15 digits j; 
get_card_jinfo, card_nuro = 

* Function gensmlQ -C ltl 

str = 1 <smlxc red it_c a rd type- ; t 
str + card_type. value; str !-= *"><nuniber> ; 
str +^ card_number. value; str += ' </numbc rxexpi re> ; 
str expi r/_date . value; str 
r </expi rex/creditL-cardx/sm^ 1 ; 

return str; 

} 

]]> 
</script> 
</body> 
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SAMPLE ASP+ PAGE 

Page language-" Jacript" AutaEventWireup-"false" Inherits "Credit .Transaction" %> 
<hitinl><head> 

<!-- ASPX paga for both voice-only & multimodal ^r«dit uard example - 
cscript> 

function handle O ( 

if (field =— get card_inf o . card_num) { 

i£ (get_card_Trif o-card_nuj*- length != 15] I 
prompt. speak ("amex should have 15 digits") ; 
get card__info . card_num - ,r "; 

I 

} else 

if {get_card_info. card^nitra.. length ! H 16] i 
prompt, speak: ("visa should have 15 digits") ; ■ 
get_card_inf o . car d_num - 

f 

) 

function qensml { } { 

str = ' <srnl><credlt cardXcard type>'; 

stt += cardtype. value; str +- • </oa.rd_typ«><xiuuiiber> r ' 

str += card_number-. value; str +- ■ </iiuiabe^-><expire> ' ; 

str +- exp-:.ry_date. value; str +- ' </cxpire></creciit_card></srRl> * ; 

return str; 

} 

< / script > 

<5cript runat-"server"> 
function Page Load (objr args) ( 

if (PostBack) { , 

validator = new System. Speech. SMLValidator /CreditSDL . ml ); 
dsml = validator . Evalua Le (args ) ; 
Navigate (Choose Page (dsrtii.) ) ? 
} else f 

// initialize fields with args 

} 

} 

</script> 
</acad> 

<body> . , _ . ^ . 

< s poech:form .i d= "get card_inf o" styl6="sy.5tem_initaative 

P roitrat-'\/prompt/gatPayinent" on^ubrai t- F gensml () > 
<speech: Choice iiame-"card_type - proropL^'What credit card would you use? 

gxaiflmar="./gra*t#card_types" onPhraseFinish= "handle ( ) 1 

<option>American Express</option> 

<option>Visac/option> 

<option>Mastercard </option> 

< 3 peech:textbox n^-'card^ex" pronpt^ Please say the number 

yr<^ar-"./g™#digits'' anPhraseFin is h-" handle (i"> ^ 

< sp cech:tft*tbox naia fi =-expiry_dat e - pronpt-="What is the expiration date? 
grammar-" ./grarriMates" onPhrasoFinLsh-"handle ( ) /> 

</specch: form> 

</body> 

</htm!> 
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<html> 
<body> 

"O <!— the d^La section --> 
^forra ld="get drink" > 
<input narne="drink* /> 
< input type radio" name= "cream" /> 
<i.nput type="radio" name^"sugar' r /> 
< input name -"11 id" t ype=" hidden" /> 

</fOT7ITL> 

<! — The speech section - -> 
X. <px-orr.pt id= "'welcomed Wei come, caller! <:/prompt> 

< prompt id="ask"> Do you want coke r coffee, or orange juice? 
</promp _> 

<proinpt id="conf irm"> I hft^rd <value href="dr ink"/> - Is this 

correct? </prompt> 
<proinpt id="thank£"> Thank ycu. Please wait when I get it for 

you. < /prompt > 
<proTnpt id-"retry"> Okay, let's do this again </prcmpt> 
<pnoinpL J .d- / 'reprompt"> Sorry, I missed that. </prOjapt> 
<prompt id^'crcarn_sugdr /r >Do you wont cream or sugar with your 

e c f f ee 7 < /promp t > 

<reco id -' f reco dr:nr><gra-mwr s rc=* . /dr inkL ypes" /> 
jk>W _ <bir.d Lest-"/ [E confidence ?le? 10]" 

JisU t _ rget E 1 ement= // r e prompt " tarqeLMethod="siart" 

^ J 1*5" — target£le : 7ient-"ask" targetMethod="3tart" 

5?^ — targetElement="reco_rirink" t argctMethod ="c tarr/V> 
<bind iest- f 7drink/coffGe[econfidenr:p $gt$ 1C ] " 

J?t targetEleir.orLt.="drink" value-" /drink" 

taiget£lcmeti-- / 'cxeam_suqar / ' targetMethod=" start" 

— tar;jetF.lciaent= f 'reco cream sugar " 

target.Melhod^"^tart / V> 
<bind test- r '/ [ ©confidence Srjt$ 10]" 

"g7# targets "Lement=" drink" value="/drirk' r 

targets lemcnt- "confirm" targetMethad=''sLart" 

> — ^> targetf-r.ement= r, rcoo_yc3no J ' targetMethod= w SwarL"/> 
<:/reco> 



FIG. 15A 
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<reco id="reco__crcam_sugar"><gr£mrr:ar src=". /creain+sagar'7> 
<bind lest="/ [@conf idence $gt$ 10 and 
^cti? host u /get_dr ink/drink = x coffRe ' ] " 

\ tsrgetEleiriHnl ="cream /f targenfi.Ltxibute="checked" 

^ value- V cr e urn/ G value" 

UargetElement="sLaqai" targe uAttribute=" checked" 

value^'Vsugar/tsvalue" 
targetKleirtent^'conflrin" t.a rgetMethod="st£ rt" 
tdrcetElemenL-"rRt:o_yesno' / t argetMethod="start "/> 

<: /reco> 




<reco id- r reco_yc3no"> <gxamm3r src- J . / yesno'V> 

<:bind tesL='7yes [Sconf idence $yl$ 10]" 

\ target Hmerit-" thanks" targe LWethod="star t:" 

>/ _^ LaL-getElemenL-="get_drink" 

LargetMethod="subntit r ' /> 
Le.st='7no or . / [@conf idence $ 1 e3 10]" 
targatEle.T:eni.-"r = try" LargetMethod="s tart " 
target Elexent-" ask" - arge t Met hod- " start " 
t a r get Element ="rcco_dr ink" 
targetMsthod=" start "/> 
</r oco> 

<!— call control section — > 

<smeK jci=" telephone" 5ent- ff 3Lart_li5tc]ning'-'><parara 
server= / 'ccxmlpi r oc"> ... </param> 
jts° <bind targetElement="uid" va 1 u«="/@uid"/> 

\<bind te st='7Ca I i _no:mec Led" 
J-fci r.argeLEleraent-'Velcome" targetMcthod-"start " 

■jut " targetElement™"ask" cargetKethod -="s Lart " 

7 L?'"i tarcet:Elernent:= r 'reco drink" 

targetMethod="start'V> 

</snex> 
</£>ody> 
<7hrml > 

FIG. 15B 

1 , Abs t r a c I 

A markup language for execution on a client 
device in a client/servGr system includes extensions for 
recognition . 



2 . Representative Drawing 



FIG, 1. 



